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ABSTRACT 


This research expands the modeling and simulation (M&S) body of knowledge 
through the development of an Implicit Model Development Process (IMDP). When 
augmented to traditional Model Development Processes (MDP), the IMDP enables the 
development of models that can address a broader array of effects than previously 
possible, giving users the flexibility to explore “hard-to-model” factors like space and 
cyber while gaining insight into their potential impacts on operational effectiveness. 
Specifically, the IMDP provides a formalized methodology for developing an improved 
model definition, where a broader, more holistic approach of defining a model’s referent 
is achieved. Next, the IMDP codifies the process for implementing the improved model 
definition within the operational model. This work serves as a proof of concept for the 
development of operational models that can account for and quantify External, Seemingly 
Intangible/Non-Quantifiable (ESINQ) factors and effects, and provides M&S users a new 
tool for addressing ESINQ and other “soft” factors that do not fit well into traditional 
MDPs. Einally, through the application of ESINQ-enabled meta-models, this work 
demonstrates how the improved understanding generated by the IMDP can be used to 
improve a set of operational and acquisitions decision support tools. 
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EXECUTIVE SUMMARY 


This dissertation introduces a new Model Development Process (MDP) that 
provides analysts the flexibility to represent hard-to-quantify factors and effects within 
operational models by taking a holistic approach to capturing a more robust set of 
potential contributions to a model’s referent. Hard-to-quantify factors and effects have 
represented a significant challenge, and rather than addressing these challenges, they 
have for the most part been largely ignored by model developers. And, while ignoring the 
irrelevant aspects of the real world is paramount to the success of any MDP, in an era of 
unprecedented cross boundary and cross domain interoperability, the determination of the 
line between relevant and irrelevant must be more closely scrutinized. By moving away 
from considering hard-to-quantify factors and external force multipliers as non- 
quantifiable, and rather describing such effects as External, Seemingly Intangible/Non- 
Quantifiable (ESINQ), a more accurate representation of these effects can be captured by 
model developers, one which addresses the possibility of quantifying the impacts of these 
effects in the referent, while highlighting the difficulty in doing so. 

This work expands the modeling and simulation (M&S) body of knowledge by 
formalizing a methodology to address, or bound, external dependencies and ESINQ 
factors and effects within the MDP. The Implicit Model Development Process (IMDP) 
developed in this work addresses the inefficiencies of traditional MDPs by formalizing a 
methodology for developing an improved model definition, where a broader more holistic 
approach of defining a model’s referent is codified. When augmented with traditional 
MDPs, the IMDP enables the development of models that can address a broader array of 
effects than previously possible, giving users the flexibility to explore “hard-to-model” 
and other “soft” factors like space and cyber that do not fit well into traditional MDPs, 
while gaining insight into their potential impacts on operational effectiveness. By 
providing a methodology that allows users to bound ESINQ effects within a model, a 
better and more complete representation of the Operational Environment (OE) is 
possible, which in turn yields improved models, analysis, and decisions based on that 
improved understanding. 



The fundamental issue with traditional MDPs is the inherent flaws in the 
underlying assumptions and methods for gathering data during the model definition steps, 
specifically during the development of the referents. Most traditional MDPs underwhelm 
the model definition step, devoting a relatively small portion of the overall resource 
budget to this critical step, often assuming the user will ensure an adequate understanding 
of the OE is instantiated in the referent prior to model development. Thus, the model 
definition step, for the most part, has been un-formalized, weakly defined, and lacking 
any specific detail of how to conduct model definition, offering just a simple framework 
or best practices for users. Models developed using traditional MDPs are primarily 
informed by just two contributing sources, internal contributors, which captures the key 
explicit system characteristics and effects that the modeler is interested in representing, 
and external contributors, which bound unknown contextual effects of interest like 
environmental effects, typically seen as distractors, degraders, or noise. Unfortunately, in 
the OE of today, where modem systems tend to rely heavily on the contributions from 
external force multipliers like space-based systems for the generation of internal combat 
power, this two-contribution approach to model development ignores a sizeable portion 
of the actual OE, where many ESINQ systems and effects reside. This author believes 
that the primary reason for the inability of traditional MDPs to capture an accurate 
assessment of the OE during model definition is their failure to recognize the existence of 
more than two sources of combat power in developing the referent. 

While current MDPs and Model Based Systems Engineering (MBSE) analysis 
methodologies support the development of accurate models and system definitions, 
neither can adequately address ESINQ factors and their impacts. In an age when network¬ 
centric operations of highly technical systems has become the standard, little has been 
done to fully understand or capture the dependencies that modern systems have on 
ESINQ factors to generate internal metrics of system effectiveness. Simply put, as 
modem systems continue to evolve increased dependencies on external elements, our 
understanding of those systems will continue to diverge from the ground truth because 
we cannot accurately attribute or quantify the impacts of those external elements. This 
incomplete understanding of the OE is then passed on to the model development step, 
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which relies on the quality of the model definition step. Thus, current MDPs produce 
models that are more wrong than they could/should be, and unfortunately, as the 
dependencies of modern systems on external support continue to rise, so will the 
inaccuracies of traditional models. The problem is further complicated by the fact that 
most traditional MDPs take a purely explicit model development approach, directing 
users to avoid hard-to-quantify input sources for fear of injecting subjectivity into the 
study. Thus, most traditional MDPs are ill-equipped to model effects that can be 
considered ESINQ, resulting in the majority of such factors and effects being ignored. 

In order to account for these external and ESINQ contributions, which are often 
subjective and difficult to quantify, users of traditional MDPs must acknowledge that not 
only do additional referent contributors exist, but also that they can be significant. By 
breaking away from the inflexible, closed-system approach of traditional MDPs, it should 
be possible to capture a more accurate representation of the OE in the referent. The IMDP 
complements traditional MDPs by formalizing a methodology for expanding the model 
definition step to account for external dependencies and ESINQ effects of interest in the 
referent. By focusing on improving the underlying MDPs to account for ESINQ force 
multipliers during model development, it should be possible to produce better models, 
execute better Operational Analysis (OA), create better decision support tools, and thus, 
make better and more informed decisions. 

This work demonstrates how the IMDP can be used to improve model definition 
and development, the two primary steps of most traditional MDPs, enabling users to (1) 
account for the external dependencies and ESINQ factors and effects that currently go 
unaddressed and (2) gain novel insights into the workings of the model. The IMDP 
expands traditional MDPs by formalizing a methodology to execute a more complete 
MDP, specifically with regard to expanding and improving the level of guidance and 
detail of the model definition step. Through the use of the IMDP, a more accurate 
representation of the actual OE can be implemented in the model, greatly improving the 
model’s fidelity and ability to link a system’s characteristics, including inputs from 
external dependencies and ESINQ effects, to metrics of operational effectiveness. Eigure 
ES-1 provides a description of how the IMDP augments traditional MDPs. 
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Figure ES-1. MDP 


For consistency with traditional MDPs, the author has broken the IMDP in two 
general steps: the model definition step (part A: decomposition); and the model 
development step (part B: integration). Part A of the IMDP formalizes the process for 
bounding ESINQ factors by better defining the interactions of the system with the 
external environment during model definition. Specifically, part A puts a significant 
amount of emphasis into the definition of the model, which differs from traditional MDPs 
by providing more structure to the weakly-defined model definition step as well as the 
means for accounting for both external dependencies and ESINQ effects in the model 
referent. In this step, ESINQ factors and effects of interest are identified, translated, 
normalized, and then calibrated in preparation for their inclusion within the model 
through an implicit modeling technique that utilizes surrogate factors. Part B of the IMDP 
formalizes the process for iteratively implementing the improved referent developed in 
part A using surrogate factors, and enables the calibration and modeling of external 
dependencies and ESINQ effects. Specifically, part B manipulates surrogate factors 
within the model to generate a range of responses that can be used to bound the impacts 
of these ESINQ effects on measures of operational effectiveness. It is the bounding of 
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these responses that is of significance to this work because it facilitates the development 
of meta-models capable of representing the impacts of ESINQ effects of interest on 
measures of operational effectiveness. 

While a standalone process from the IMDP, this work also provides a formali z ed 
methodology for screening and selecting an appropriate M&S package, which guides the 
user to the selection of a more appropriate M&S package, a step typically ignored in 
traditional MDPs. The combination of these three improvements to current MDPs allows 
for a more complete understanding of the system, to include external dependencies and 
ESINQ effects and their interactions with the environment within the model, as well as a 
more complete investigation of the system trade-space. By developing a methodology 
that can more accurately address ESINQ effects within a model, it will be possible to 
capture a more holistic understanding of the OE. By making operational and acquisitions 
decisions based on the performance of competing emerging systems and strategies within 
this more holistic understanding of the actual OE, we greatly increase the chances for 
success by producing a more robust system with direct traceability to metrics of 
operational effectiveness. 

The primary outcome from the execution of the IMDP was the development of 
meta-models that can quantify the impacts from ESINQ effects on metrics of operational 
effectiveness. These meta-models have a nearly limitless capacity to inform and to 
improve other tools: yet for the most part, the application of such outcomes has remained 
mostly unformalized. Differing from other MDPs, the IMDP puts a more formal effort 
into the utilization of these meta-models for use within external tools, specifically in the 
development of ESINQ-enabled operational and acquisitions decision support tools, a 
focal point of this dissertation. In this work, we show how the ESINQ-enabled meta¬ 
models can be applied to other tools to improve their ability to capture a more accurate 
representation of the OE. Two such examples are given, the first of which was the 
development of the Improved Relative Combat Power Assessment Tool (IRCPAT) which 
can be seen in Eigure ES-2. 
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Figure ES-2. IRCPAT 

The IRCPAT shown here has been instantiated to include the degraded 
Intelligence, Surveillance, and Reconnaissance (ISR) meta-model developed during the 
execution of the IMDP in Chapter IV, specifically formatted to provide a degradation 
factor with respect to Relative Combat Power (RCP). The IRCPAT is unique in that 
neither external dependencies nor ESINQ contributors to combat power were previously 
accounted for in modern operational decision support tools. Thus, the IRCPAT fills 
significant gaps in modern operational decision support tools by accounting for the 
contributions of external dependencies, ESINQ effects, and external systems on the 
generation of combat power, and facilitates a more detailed and complete analysis of the 
RCP of opposing forces. By improving its accuracy, usability, and relevance, the 
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IRCPAT will be far more useful than eurrent tools, and will have immediate relevance to 
operational planners Army-wide. 

The second example used in this work demonstrated the application of the 
outcomes of the IMDP to support the development of an ESINQ-enabled acquisition 
support tool. The power of acquisitions decision support tools to effectively link 
operational and synthesis models is clearly articulated in modem works, and provides 
users extremely flexible Trade Space Exploration (TSE) tools for investigating system 
design considerations and the resulting impacts to operational effectiveness. By applying 
the outcomes of the IMDP, specifically through the integration of the ESINQ-enabled 
meta-models, a more complete TSE tool was developed. This expansion gave TSE tools 
more utility for the user and allowed for a more accurate assessment of not only the 
operational impacts from degradation, but the potential of emerging systems to mitigate 
these impacts. The IMP TSE tool developed in this work can be seen in Eigure ES-3. 
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By using an ESINQ-enabled operational model in conjunction with a linked 
synthesis model, the resulting TSE tool can account for the external dependencies and 
ESINQ effects previously ignored in other tools, and thus, capture a more inclusive and 
accurate representation of the OE during analysis. By expanding the TSE tool using 
ESINQ-enabled models, we allow for the assessment of ESINQ effects. Thus, users can 
better assess ESINQ-based systems whose contributions are measured in terms of ESINQ 
effects, like SmallSats, something that has been difficult to do using traditional MDPs. 
When used to assess the feasibility and impact of emerging ESINQ-based systems, this 
improved TSE tool fills known gaps within the acquisitions community and provides 
users with a more detailed and complete analysis of the impact that design choices will 
have on metrics of operational effectiveness. This improvement of current TSE tools 
through the use of ESINQ-enabled models will have immediate operational relevance to 
acquisitions planners, and should provide them a more robust capability to evaluate 
design decisions regarding emerging ESINQ systems. 

The ability of the IMDP to support the development of better models facilitates 
the secondary contributions of this work. When these models are used as foundational 
elements in other decision support tools, the overall accuracy of these tools is improved. 
This improvement is largely due to the underlying models’ increased accuracy at 
representing the OE and their ability to account for a larger number of potential 
contributions to RCP, specifically through addressing the external dependencies and 
ESINQ effects so routinely ignored in traditional MDPs. The IMDP presented in this 
work gives users the ability to loosely quantify or “bound” the impacts from ESINQ 
factors and effects on measures of operational effectiveness. Thus, it becomes possible to 
conduct quantifiable system analysis capable of assessing the contributions of ESINQ 
factors and effects (like space-based capabilities) to friendly combat power. By following 
the IMDP presented in this work, a better evaluation of the OE can be made, which will 
allow for more informed operational and acquisition decisions regarding the allocation of 
resources, which will, in turn, greatly improve the OA activities of the M&S community 
at large. 
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I. INTRODUCTION 

To rely on rustics and not prepare is the greatest of crimes; to be prepared 

beforehand for any contingency is the greatest of virtues. 

—Sun Tzu, The Art of War (quoted in Jackson 2014) 

The U.S. Armed Forces may be the most capable military force the world has ever 
seen, yet this capability depends heavily on the use of technology. It is the integration of 
technology into all aspects of operations that has allowed the U.S. to gain and maintain a 
significant tactical advantage over our adversaries. However, to fill the gaps left vacant 
by its shrinking force while maintaining its current tactical advantage, our military must 
increase its dependencies on force multipliers like space-based capabilities. Thus, for 
example, as the Army continues to transition under the new Army Operating Concept 
(AOC) into a smaller and more capable force, force multipliers will be expected to 
contribute an even larger portion to the overall combat power. Unfortunately, this 
transition does not come without risk. As stated in U.S. Policy, in today’s Operational 
Environment (OE) “potential adversaries are seeking to exploit perceived U.S. space 
vulnerabilities. As more nations and non-state actors develop counterspace capabilities 
over the next decade, threats to U.S. space systems and challenges to the stability and 
security of the space environment will increase” (Department of Defense [DOD] 2011a, 
3). Clearly, our adversaries are developing capabilities that aim to reduce U.S. tactical 
advantages. To ensure this does not come to pass, the U.S. Armed Eorces must follow the 
teachings of Sun Tzu and “prepare.” 

A. MOTIVATION 

In modern warfare, the availability of space-based capabilities is often taken for 
granted. These capabilities are so thoroughly ingrained into operations that one can argue 
the U.S. Military has become too dependent on the tactical advantages these capabilities 
bring to the warfighter. Since the first significant use of space systems in support of 
combat operations during Desert Storm, the Department of Defense (DOD) has steadily 
increased its investment in space support capabilities. Eooking back, it is evident that 


1 



the Army has long recognized the positive value of space capabilities in 
enabling modern warfare. Satellite Communication (SATCOM) allows 
forces to operate over extended ranges; the Global Positioning System 
(GPS) delivers precise positioning, navigation, and timing; and space- 
based Intelligence, Surveillance, and Reconnaissance (ISR) systems 
provide unprecedented situational awareness of adversaries. (U.S. Army 
2014a, 1-2) 

But the Army is not alone in its leveraging of space-based capabilities, as the use 
of these capabilities has proliferated to such an extent that “military, civil, and 
commercial sectors of the U.S. are increasingly dependent on space capabilities, and 
this dependency is a potential vulnerability as space becomes increasingly congested, 
contested, and competitive” (Joint Chiefs of Staff 2013,1-2). Unfortunately, as the U.S. 
military transitions under guidance like the new AOC, this dependency is sure to grow, 
and adversaries will seek to take advantage of our dependency. In the OE of the future, 
the space and cyber-space domains will begin to take a dominant role in warfare, driving 
state and non-state actors to “invest in capabilities to protect their access and disrupt or 
deny access to others,” and thus, it must be expected that “Army units will have to 
operate with degraded communications and reduced access to cyber and space 
capabilities” (U.S. Army 2014b, 11-12). Thus, the U.S. Military finds itself in a 
precarious position, stuck between opposing requirements of a shrinking force, 
operational readiness, and a threat. 

These requirements force the U.S. military to shift combat power from local to 
external sources, which include space-based systems where, as indicated, known 
vulnerabilities exist. To say the U.S. military is prepared for the impacts of outsourcing 
combat power is optimistic. Most doctrinal sources point to the military being neither 
adequately trained nor equipped to effectively operate in a Denied, Degraded, and 
Disrupted Space Operating Environment (D3SOE) as outlined in the new AOC. The U.S. 
military has become dependent on the advantages that space brings the warfighter, yet it 
has done little to understand the potential risks of this dependency or to prepare for when 
we lose that advantage. The area of Operational Analysis (OA) is of most concern to this 
dissertation, specifically with regard to the inability of OA to provide operational and 
acquisitions decision makers with the information needed to make better decisions. 
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One area of OA where the U.S. military fails to adequately prepare for operations 
in a D3SOE is in operational planning. Today, commanders and staffs at all levels of the 
DOD have an unrealistic and sometimes non-existent expectation of the impacts to 
operations from a D3SOE. This is concerning because not only do we know that 
operations in a D3SOE is inevitable, but we have been told in National Policy to plan for 
it. As directed in the DOD Space Policy (2012), combatant commanders shall “develop 
and exercise operational concepts as well as tactics, techniques, and procedures to 
continue operations and achieve assigned national security objectives in an environment 
in which space capabilities have been degraded or denied” (11). Yet even with national 
directives demanding preparation, the author’s operational experience show that most 
organizations routinely underestimate the threat from operations in a D3SOE. While 
some space planners are attempting to mitigate this threat, they often face significant 
obstacles from their own organizations, primarily due to the lack of any quantifiable 
source of information to justify their claims. Without such information, it is nearly 
impossible to successfully advocate for preparedness for operations in a D3SOE. 

Another area of OA where the U.S. military fails to adequately prepare for 
operations in a D3SOE is in space Research and Development (R&D) and Acquisitions. 
Currently, as each service identifies emerging space capabilities, it allocates resources 
based on the potential utility they have in support of operations. There are two major 
issues with allocating resources in this way. Eirst, it is done through only the most basic 
of heuristics, dependent on a qualitative assessment from various Subject Matter Experts 
(SMEs). Second, there is no common definition of what utility is or what it means to the 
warfighter. These issues significantly decrease a program’s chance of success because 
there is neither a common nor a quantifiable metric to describe a system’s contribution to 
the supported unit’s overall operational effectiveness. How can we select the most 
feasible, mission-effective, and fiscally sound mix of alternatives if we have no means to 
quantify their contribution to operational effectiveness or to compare alternatives? As 
GEN Perkins said in his comments at the Naval Postgraduate School (NPS) on December 
9, 2014, “In budget-constrained times it is important to have clarity of purpose” (Perkins 
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2014). To achieve this clarity, the U.S. military OA decisions of the future must be based 
on accurate and quantifiable information rooted in Measures of Effectiveness (MOE). 

By using quantifiable metrics to screen out less operationally effective programs, 
resources can be better allocated to programs with the highest potential utility to the 
warfighter. While not all inclusive, the author believes that the U.S. military’s failure 
during OA, specifically operational planning and acquisitions, is the primary contributor 
to the lack of preparedness for mitigating the effects of a D3SOE. This failure is likely 
due to the lack of two key capabilities: 

1. A means to quantify the contributions of force multipliers like space-based 
systems and impacts from counter-space systems on operational 
effectiveness. 

2. Operational and acquisitions decision support tools that can better 
articulate these impacts to leaders and decision makers as they plan for 
operations in a D3SOE. 

By focusing on improving the OA process, supported by more accurate decision 
support tools, it is possible to provide military leaders with quantifiable information 
regarding force multipliers, and thus, should facilitate better decisions regarding 
preparation for operations in a D3SOE. 

B. PROBLEM 

Current OA methodologies lack decision support tools that can quantify the 
impacts from operations in a D3SOE. Thus, DOD space planners simply have no way to 
inform staff and leadership regarding the quantifiable impacts to friendly combat 
effectiveness from operations in a D3SOE. With over twelve years of experience as an 
Army space planner, it is the author’s belief that this inability to inform the staff is 
largely due to the misconception among DOD planners that these effects are non- 
quantifiable or intangible, and thus outside or external to the scope of planning. While the 
utility of space systems is well understood, as well as the potential threats from adversary 
counter-space capabilities, neither can be easily measured, and thus, they are typically 
ignored during the Military Decision Making Process (MDMP) because they are 
considered non-quantifiable. Unfortunately, it is this perception of space-based 
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capabilities and other external force multipliers as non-quantifiable that is likely at the 
root of the problem. While ESINQ effects may be difficult to quantify, they are by no 
means non-quantifiable; up to this point, there has simply been no formal effort within 
the community to quantify them. This is not just an DOD problem either; the Modeling 
and Simulations (M&S) community as a whole typically either ignores the hard-to- 
quantify external dependencies and effects, aggregates them into the context, or 
implicitly models them in other modeled aspects. While models are by definition 
abstractions of reality, capturing “the essential aspects of a simuland to be represented in 
a model or simulation while excluding those aspects that are not relevant to the purpose 
of the model or simulation” (Modeling and Simulation Coordination Office 2017), to 
make the determination of relevance without any logical rigor is potential dangerous. As 
described by Johns Hopkins University Applied Physics Lab (JHU APE), “the simuland 
is often casually referred to as the ‘real world’ or as reality, actuality, or truth. However, 
no simuland actually achieves equivalence with the ‘real world’” (Johns Hopkins 
University Applied Phyisics Lab [JHU APE] 2017). In reality, simulations are an 
“abstraction drawn from the sum total of what is known, assumed, or projected about the 
simuland, called a referent” (JHU APE 2017), which are often codified in terms of 
requirements and an operational concept. Model developers m understand the potential 
harm that can come from a model whose referent ignores potentially significant 
contributions to the outcome of the model. While fidelity addresses the accuracy of the 
model in representing the referent, how accurate can the model be if the referent is 
incomplete? Thus, while ignoring the irrelevant aspects of the real world is required in 
traditional Model Development Processes (MDP), in an era of unprecedented cross 
boundary and cross domain interoperability, the determination of where the line between 
relevant and irrelevant is drawn must be more carefully scrutinized. By moving away 
from considering external force multipliers like space-based effects as non-quantifiable, 
and rather describing such effects as External, Seemingly Intangible/Non-Quantifiable 
(ESINQ), a more accurate representation of these effects can be captured, one which 
addresses the possibility of quantifying the impacts of these ESINQ effects in the 
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referent, while highlighting the difficulty in doing so. For consistency in this document, 
the following four statements will be used to characterize an ESINQ effect. 

• ESINQ effects impact the outcomes of the model. 

• ESINQ effects are not modeled (external to the system/model boundary). 

• ESINQ effects are difficult to quantify rather than non-quantifiable. If they 
are truly non-quantifiable they are not ESINQ. 

• ESINQ effects are ignored without scientific rigor. If factors can be 
discarded following a logical assessment, they are not ESINQ; they have 
simply been determined to be insignificant for the purposes of the study. 

It is the author’s belief that the inability of current OA processes to account for 
ESINQ effects is due to the inability of traditional MDPs to recognize the existence of 
more than two contributors to a model’s referent. Eigure 1 provides the author’s 
interpretation of the contributors to the referent of traditional MDPs. 



Eigure 1. The Traditional MDP Referent 
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The blue cloud in Figure 1 represents a general view of the real world, which 
includes an infinite amount of modeling potential. In this example, the system is a generic 
model that, through abstraction, tries to capture some small piece of the overall 
environment. To do so, the model is developed using traditional MDPs, whose referent is 
primarily informed by just two contributing sources. The first source is internal 
contributors, which capture the key explicit system characteristics and effects that the 
modeler is interested in representing. The second source is external contributors, which 
bound unknown contextual effects of interest, typically seen as distractors, degraders, or 
noise. The intent of the model (the green oval) is to capture enough key aspects of the 
system in the context of the physical world to provide an approximation of the system of 
interest, which includes some of the interactions with the environment. Because of this 
focus, the majority of the model’s inputs are based solely on the system of interest, 
specifically the quantifiable details derived internal to the system, with only minimal 
accounting of the referent of other contextual factors. Unfortunately, in the OE of today 
where modern systems tend to rely heavily on the contributions from external force 
multipliers—like space-based systems—for the generation of internal combat power, this 
approach can potentially ignore a sizeable portion of significant input factors during the 
establishment of the referent. These sources, like external dependencies and ESINQ 
factors and effects, often reside in the area of the OE (the blue cloud) that is typically 
ignored during the development of the referent used in traditional MDPs. In order to 
account for these contributions, which are often very subjective and difficult to quantify, 
traditional MDPs must acknowledge that not only do additional referent contributors 
exist, but also that they can be significant. By breaking away from the inflexible, closed 
system approach of traditional MDPs, it should be possible to capture a more accurate 
representation of the OE in the referent, as seen in Eigure 2. 
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Figure 2. The Expanded MDP Referent 


By acknowledging that a more refined referent can be achieved by sub¬ 
categorizing the contributions of both internal and external contributions, it will be 
possible to form a more holistic understanding of the OE, the portion of the real world of 
interest that defines the context for the referent used during the MDP. This improved 
understanding can first be achieved by delineating between the different sources of 
internal contributions to the referent. By separating a model’s input sources into two 
areas—a system’s internal contributions and a system’s internal contributions that are 
externally dependent—it is now possible to decouple any dependencies the system has on 
external resources. Unfortunately, due to the focus on explicit modeling techniques, most 
traditional MDPs either ignore these external dependencies or aggregate them with 
internal contributions during the establishment of the referent. Thus, the two internal 
sources cannot be observed independently, resulting in the inability of most MDPs to 
account for the impacts that external dependencies have on internal metrics of operational 

effectiveness. Once separated, users can degrade the portion of the system’s contributions 

8 










that is dependent on external support, which facilitates the quantification of the 
operational impacts due to a D3SOE, a key outcome of this work. 

Next, this understanding can be further improved by delineating between the 
different sources of external contributions to the referent. By separating the external 
contributors to models into two areas, contextual contributions and ESINQ contributors, 
it is possible to highlight a fourth area of potential contributions to the referent that is 
currently unaccounted for within traditional MDPs. The key difference between the two 
is that contextual factors are by definition unknown and/or uncontrollable, but ESINQ 
factors are not; they are both known and controllable, though they are often extremely 
difficult to quantify. By recognizing ESINQ factors and effects as potentially significant 
model contributors, and acknowledging them as known, albeit difficult to quantify, the 
inclusion of ESINQ systems like space-based systems into the referent is facilitated. This 
more inclusive process supports the quantification of the operational impacts from these 
systems, which is another key outcome of this work. While this improved understanding 
of the OE does not change the fact that traditional MDPs fail to adequately account for 
either external dependencies or ESINQ factors and effects in the referent, as seen by the 
lack of overlap of the model with the four potential contribution sources in Eigure 2, it 
does set the conditions for a more inclusive MDP capable of accounting for them. 

C. RESEARCH QUESTIONS 

The primary question driving this research is as follows: Can current MDPs be 
modified to account for ESINQ effects within models? If so, can these ESINQ-enabled 
models be used to quantify the operational impacts from ESINQ effects? With these 
questions in mind, the aim of this work is to identify how M&S and Model Based 
Systems Engineering (MBSE) can be used to address observed gaps within operational 
and acquisitions planning by supporting better decision making through the quantification 
of ESINQ effects. By providing better information to decision makers, information that is 
capable of capturing a more accurate understanding of the OE, a more robust solution set 
should be possible. In constructing this question, the following supporting questions were 
also identified for their potential in supporting OA decisions: 
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1. What models are capable of representing the contributions from external 
dependencies and ESINQ effects, and to what level of resolution? 

2. How can MBSE and M&S be used to aid decision makers within the OA 
fields to filter, design, configure, compare, select, and allocate resources to 
emerging capabilities? 

3. Can an emerging system’s physical and operational models be linked 
through design responses? Can this interaction be captured in a meta¬ 
model that is tied to metrics of operational effectiveness? 

4. Can ESINQ meta-models be used to improve current operational and 
acquisitions decision support tools to provide a more robust solution with 
regard to operational effectiveness? 

In addressing the primary and supporting research questions, this dissertation fills 
some significant gaps in the ability of current MDPs to capture ESINQ effects. By 
providing a methodology that can bound ESINQ effects within a model, a better 
representation of the OE in the referent will be possible. This more complete 
understanding of the OE will improve the models developed through current MDPs; will 
improve the assessments generated from MBSE analysis methodologies; and support 
more informed decisions regarding both the use and allocation of resources. Together, 
these improvements will better address the gaps described in the motivation and problem 
sections, and support U.S. military space professionals to make more accurate and 
informed decisions as we “prepare” for operations in a D3SOE. 

D. PRIMARY AND SECONDARY CONTRIBUTIONS 

To address the inability of traditional MDPs to account for the contributions from 
external dependencies and ESINQ effects, an improved MDP is needed that can enable 
the inclusion of all four of the referent contributors. This work expands the M&S body of 
knowledge through the development of a formalized methodology to account for, or 
bound, ESINQ factors and effects within the MDP. The intent is to address the lack of 
synergy in traditional MDPs by developing an Implicit Model Development Process 
(IMDP) as well as a set of operational and acquisitions decision support tools to support 
the quantification of impacts from ESINQ effects within a model. This expansion 
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improves the utility of current decision support tools, and results in a better understanding 
of the impacts of ESINQ effects on metrics of operational effectiveness. 

1. Primary Contribution 

This work demonstrates how the IMDP can be applied to improve model 
definition and development, the two primary steps of most traditional MDPs (the third 
step is model analysis). This improvement enables users to gain novel insight into the 
workings of the model and account for the external dependencies and ESINQ factors and 
effects that currently go unaddressed in traditional MDPs, and while not discussed in 
detail here, a more in-depth discussion of the strengths and weaknesses of traditional 
MDPs is found in Chapter II. Because traditional MDPs tend to underwhelm the model 
definition step, devoting a relatively small portion of the overall resource budget to what 
most systems engineers would consider the critical step, the models produced based on 
this definition are often more inaccurate then they could be. This problem is further 
complicated by the fact that most traditional MDPs take a purely explicit model 
development approach, directing users to avoid hard-to-quantify input sources for fear of 
injecting subjectivity into the study. Thus, most traditional MDPs are ill-equipped to 
model systems or effects that can be considered ESINQ. The IMDP complements 
traditional MDPs by formalizing a methodology for expanding the model definition step 
to account for ESINQ effects of interest in the referent. While the IMDP can serve as a 
standalone MDP, it was developed to augment traditional MDPs in order to improve the 
ability of the model to capture the actual OE. Specifically, the IMDP focuses on 
including the impacts from external dependencies and ESINQ factors and effects. 
Through the use of the IMDP, a more accurate representation of the OE can be 
implemented in the model, greatly improving the model’s fidelity and ability to link a 
system’s characteristics, to include inputs from external dependencies and ESINQ effects, 
to metrics of operational effectiveness. To demonstrate, Eigure 3 gives a general 
description of how the IMDP can be applied to a generic MDP to produce an ESINQ- 
enabled model, capable of quantifying the impacts of ESINQ systems and effects on 
operational effectiveness. 
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Figure 3. Implicit Model Development Process 


Differing from traditional MDPs, part A of the IMDP puts a significant amount of 
emphasis into the definition of the model. In this step, ESINQ factors and effects of 
interest are identified, translated, normalized, and then calibrated in preparation for their 
inclusion within the model through an implicit modeling technique that utilizes surrogate 
factors. In part B, the surrogate factors can be instantiated within the model and then 
manipulated to generate a range of responses that can be used to bound the impacts of 
these ESINQ effects on measures of operational effectiveness. It is the bounding of these 
responses that is of interest to this work because it facilitates the development of meta¬ 
models capable of representing the impacts of ESINQ effects of interest on measures of 
operational effectiveness. These meta-models can in turn be used to improve the accuracy 
of operational and acquisitions support tools, giving planners the means to provide 
decision makers with the quantifiable data necessary to make better decisions. 
Additionally, because the IMDP enforces traceability to a reference tool throughout the 
process, partial validation of the ESINQ-enabled model is achieved. Through the use of 
the IMDP, a more inclusive model can be developed, one capable of accounting for a 
larger portion of the OE in the referent, as seen in Eigure 4. 
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E: "ernal Contributors 



Figure 4. The IMDP Referent 


Through the use of the IMDP, the model (green oval) can capture a larger portion 
of the OE through the use of pre-existing surrogate factors within the model to represent 
the ESINQ effects and external dependencies of interest. This ESINQ-enabled model has 
two significant contributions. Eirst, it allows us to quantify through meta-models the 
operational impacts due to ESINQ factors and effects, which can be used to support 
operational support tool development. This work puts a more formal effort into the 
utilization of the operational model meta-models for use within external tools, 
specifically in the development of operational decision support tools. Second, when used 
in conjunction with a synthesis model of a system’s physical components, a more 
thorough Trade Space Exploration (TSE) can be conducted during design because the 
underlying operational model can more accurately represent the OE. This is critical for 
acquisitions professionals executing analysis of emerging systems whose operational 
impacts often reside in areas considered as ESINQ, like space-based systems. Through 
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this use of the IMDP, it becomes possible to conduct quantifiable system analysis capable 
of assessing the contributions of ESINQ factors and effects (like space-based capabilities) 
to friendly combat power; the impacts to operational effectiveness from adversary use of 
ESINQ factors and effects (think counter-space activities); and the capabilities of non- 
traditional/ESINQ-enabled systems (emerging space systems) to mitigate adversary 
threats. This capacity will greatly improve the OA activities of the United States, as well 
as improve the quality of the decisions resulting from the analysis. 

2. Secondary Contributions 

This dissertation aims to expand the overall capability of current MDPs and to 
improve the accuracy of models through a formalized IMDP that can account for ESINQ 
factors and effects. In accomplishing this, five secondary contributions are made that 
address many of the problems described earlier in this chapter. These included the 
development of an ESINQ-enabled combat model, an operational support tool, an 
acquisitions support and TSE tool, the development of an M&S Selection and Screening 
Methodology (MSSSM), and the expansion of M&S and MBSE processes. A depiction 
of the IMDP primary and secondary contributions can be seen in Eigure 5. 
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The primary contribution of this work is generalized, and supports the expansion 
of the referent of traditional MDPs to account for ESINQ factors and effects. The 
secondary contributions on the other hand are more specific, and focus on the motivation 
for this work, which is the quantification of the impacts from space-based capabilities and 
operations in a D3SOE on operational effectiveness. A brief description of each of these 
secondary contributions is provided, and will be discussed in more detail throughout this 
dissertation. 


a. ESINQ-Enabled Operational (Combat) Models 

Space planners do not currently have access to an operational model that can fully 
demonstrate the impacts of a D3SOE on operational effectiveness of the operational 
force. Without a quantifiable and consistent means to evaluate a system’s impact on 
higher level MOEs it is not possible to compare mitigation strategies, and decisions made 
in this manner will produce outcomes that have little traceability to actual MOEs. The 
most significant outcome from the IMDP developed in this dissertation is in the creation 
of ESINQ-enabled entity models that more accurately represent modem operations as 
outlined in the new AOC, to include dependencies on space-based capabilities and the 
impacts from counter-space capabilities. Because entity level models (physics based) are 
uncommon at the tactical level, the outcomes of this work will support the translation of 
ESINQ effects to improve the aggregate models and tools commonly used at the tactical 
level. This will provide a much needed tool for planners to integrate D3SOE activities, 
allowing them to quantify operational impacts from a D3SOE and better support the U.S. 
military as it “prepares” for an uncertain future. The ESINQ-enabled operational models 
will be discussed further in Chapter IV. 

b. ESINQ-Enabled Operational Support Tool 

The tool that is most widely used to support Army operational planning is the 
Eorce Ratio Calculator (ERC), an Excel based model that estimates the Relative Combat 
Power (RCP) of two opposing forces based on mission set and force size. Unfortunately, 
“force ratios do not include the environmental and human factors of warfare” 
(Department of the Army [DOA] 2005, 3-31), nor do they account for external 
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dependencies or contributions from sources of combat power outside of the system 
boundary. Thus, Army Course of Action (COA) development cannot accurately capture 
the RCP of the opposing forces, forcing leaders to make decisions with an incomplete 
and inaccurate assessment of the OE, which can have unforeseen costs in lives, time, and 
money. Therefore, the FRC tends to underestimate the capabilities and vulnerabilities of 
the forces being modeled and as noted by Zanella in his thesis (2012), produces RCP 
ratios where “the estimated ratios are probably too low” (8). Using the ESINQ-enabled 
model and the output from a large scale Design of Experiments (DOE), this dissertation 
developed an improved version of the ERC that provides planners the means to quantify 
the impacts from ESINQ factors and effects. By highlighting space dependencies and 
threats that were ignored in the past, a more accurate assessment of RCP can be achieved. 
The ESINQ-enabled operational support tool will be discussed further in Chapter V. 

c. ESINQ-Enabled Acquisition Decision Support Tool 

This dissertation developed an ESINQ-enabled acquisition decision support tool 
for use by space R&D and Acquisitions communities to conduct TSE. This tool allows 
decision makers to visualize and explore the design trade space of emerging space 
systems earlier in the design life cycle, and gain a better understanding of the systems and 
their interactions within a more accurate representation of the OE prior to resource 
allocation. By linking operational and physical design feasibilities to a quantifiable 
assessment of a system’s impacts to operational effectiveness, imbedded in an ESINQ- 
enabled representation of the OE, decision makers can make better informed decisions 
regarding the resource allocation of competing capabilities. The ESINQ-enabled 
acquisition decision support tool is discussed further in Chapter V. 

d. M&S Screening and Selection Methodology 

In most MDPs and analysis methodologies, users are assumed to use an 
appropriate M&S package for the purpose of the study and the analysis they are 
interested in conducting. Thus, no formalized techniques have been defined to support 
M&S screening and selection. Unfortunately, there are two major issues with this 
assumption. First, most users do not understand the breadth or depth of available M&S 
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resources. Second, most users typically select the M&S package they are most 
accustomed to, without consideration for the utility of that package for the specific 
problem. This is a significant disadvantage of current MDPs, and results in the majority 
of analysts selecting non-optimal M&S packages. This dissertation will develop a 
formalized MSSSM that will support a better investigation of potential M&S packages, 
and by doing so, increase the likelihood of selecting a more appropriate M&S package 
for use. The MSSSM will be discussed in Chapter III. 

e. Expanding the Acceptability of M&S and MBSE 

MBSE is an emerging branch of Systems Engineering (SE) which emphasizes the 
importance of modeling through the formalized application of M&S throughout the SE 
process. It is a capability-based decision making process which ties OA decisions to 
quantifiable impacts on metrics of operational effectiveness, and thus allows for better 
decisions regarding the selection of alternative designs. This work expands the 
understanding, use, and the acceptability of M&S and MBSE by improving current 
MDPs through the execution of the IMDP, as well as applying it for the first time to the 
space systems operational and acquisition fields. The expansion of the acceptability and 
use of M&S and MBSE is discussed throughout this dissertation. 

E. METHODOLOGY 

Eor this dissertation, the author used quantifiable and sequential mixed methods 
research using MDPs, MBSE, and effects-based decision making. By synergistically 
applying these processes and methodologies to a common problem, it will be possible to 
better align early conceptual design modeling to metrics of operational effectiveness. 
Thus, through the development of a IMDP and a set of improved operational and 
acquisition support tools, this work enables better informed decision making by providing 
leaders with more robust information regarding the impacts of ESINQ factors and effects 
(space systems) on operational effectiveness of the warfighter in a D3SOE. 
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1 . 


Modeling Outline 


In achieving the outcomes described in this chapter, three models were developed 
to generate the required data and tools. These models included an operational model, a 
physics based model that captures the operational requirements and constraints of 
systems and their impacts to operational effectiveness; a synthesis (physical system 
design) model, to capture the physical requirements and constraints of emerging space 
systems; and a linked model, where meta-models of both the operational and synthesis 
models are linked via common response factors for use in decision support tools. This 
modeling outline can be seen in Figure 6. 


Modeling Tier 

Tier 1: Force-on-Force 
Operational Effectiveness 
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Figure 6. Modeling Outline 


The left side of the figure shows the four tiers of operational model development, 
the right side shows the synthesis modeling, and on the bottom we see the linked model. 
The primary contribution of this work was accomplished in Tier 2, where the IMDP 
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supports the development of a physics based operational entity model that can account 
for ESINQ factors and effects through the use of surrogate factors and then quantify the 
impacts of these effects on operational effectiveness. This contribution supports the 
development of all but one of the secondary contributions listed in this work, including 
the development of an operational decision support tool, an aggregate model that is 
improved through the application of ESINQ-enabled meta-models. The remaining tiers, 
the synthesis model and the linked model are used to support the development of the 
acquisitions decision support tool. 

The operational model will be developed incrementally in four tiers. In Tier 1, a 
model will be created that captures the best possible representation of the scenario within 
the OE as possible, without consideration for ESINQ factors or effects. In Tier 2, the 
model will be expanded to include the ESINQ factors of interest, which in this work 
captures space support requirements like dependencies on GPS, SATCOM, and ISR. In 
Tier 3, this model will be expanded again to include adversary counter-space capabilities. 
Einally, in Tier 4, the model will be expanded to include emerging space capabilities like 
Small Satellites (SmallSats) and High Altitude Atmospheric Satellites (HAAS) in order 
to capture their ability to mitigate the effects of adversary counter-space capabilities. This 
model, executed in conjunction with a DOE, will produce the data needed to create a 
meta-model that can be used as a surrogate in other models and tools. 

The synthesis model developed in Tier 4 is actually the most complex of the 
models developed in this work. Because physical characteristics impact Measures of 
Performance (MOP), and user inputs, requirements, and constraints impact system 
characteristics, the development of a model that can produce a representation of the trade 
space of emerging systems while maintaining the flexibility to allow user input is 
extremely complicated. To reduce the complexity of this process, this work expands upon 
the thesis work of Ordonez (2016), and use a modified version of his SmallSat synthesis 
model for use in this work. Eike the operational model, this will be used in conjunction 
with a DOE to produce a good fitting meta-model that represents the user settings in the 
synthesis model with the expected impacts on metrics of operational effectiveness. 
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The linked model, while simply the combination of model representations, or 
meta-models of the operational and synthesis models, will be the final model developed 
in this work. The purpose of this model is to provide the framework for the development 
of a visual decision support tool that will allow users to explore the system design trade 
space of system alternatives. This tool will highlight how user requirements, constraints, 
and decisions affect a system’s impact to operational effectiveness. Unlike other methods 
of evaluating alternative systems, this tool will allow for a complete and thorough 
investigation of system tradeoffs that can simultaneously screen alternative designs for 
both operational and physical design feasibility. 

2. Research Scope 

National Policy and Doctrine establishes the need for capabilities, methodologies, 
and Tactics, Techniques and Procedures (TTP) to prepare for and mitigate the operational 
impacts of adversary counter-space activities. This research focuses on developing an 
IMDP and a set of operational and acquisitions decision support tools that will support U.S. 
military operations and acquisitions decision makers to make better decisions. By 
providing quantifiable data that links metrics of operational effectiveness to physical design 
parameters, decisions regarding the resource allocation of emerging space systems should 
be improved. Yet this is a difficult problem, and rather than attempting to accomplish such 
a complex undertaking, this research was scoped in the following ways to reduce the 
overall complexity of the problem. 

First, this work serves as a proof of concept, and establishes a methodology for 
accounting for ESINQ factors and effects within the MDP. By avoiding the highly detailed, 
high resolution, and Verification, Validation, and Accreditation (VV&A) M&S packages 
typically used within the DOD, the complexity of model development was significantly 
reduced. This allowed the research to investigate a broader range of research questions, 
significantly increasing the contribution of this work to the overall body of knowledge. 
Additionally, to further reduce the complexity of this investigation, the scope of this work 
focuses on the tactical level, specifically the Brigade and below. Thus, the models 
developed in this work were relatively small, ranging in size from 30 to a few hundred 
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agents, and used relatively short conflicts, ranging from minutes to hours. While the direct 
results of this work may not be adequate for long term use by the U.S. military, it can be 
used to fill known capability gaps by providing products that have immediate applicability 
to current operations. 

Next, because one of the secondary contributions of this work was the 
development of an acquisitions decision support tool to help visualize the design trade 
space prior to resource allocation, this work focused on the conceptual design phase of 
the SE process. Thus, the majority of the synthesis modeling done in this work was 
intentionally low resolution. In order to maintain the flexibility of the tool to allow users 
to fully explore the design trade space, it must be capable of accepting a wide range of 
user inputs, to include requirements, constraints, thresholds, and goals. By developing a 
low resolution synthesis model which captures only the dependencies and interactions of 
the major design factors, a more robust model for use in early TSE was produced. 

Easily, because another of the secondary contributions of this research was the 
development of an operational decision support tool that could provide utility to 
operational space planners, this work focused on solutions that were easy to use, could be 
rapidly developed (minutes not hours), were free and openly available at the unclassified 
level, stochastic, and constructive in nature. These choices were made by the author in 
order to maximize the extent to which the products developed in his work could support 
operational space planners, specifically focusing on mission planning, training, and 
exercises, where the most impact can be made in preparing for operations in a D3SOE. 

F. ORGANIZATION 

This dissertation is organized into six chapters. Chapter II reviews the prior work 
and literature regarding this research, focusing on the current threat, mitigation strategies, 
the Army’s use of space, the MDMP, and gap analysis. Chapter III outlines the primary 
contribution of this dissertation through a detailed presentation of the IMDP. Chapter IV 
includes a complete demonstration of the IMDP for a space specific problem. In this 
chapter, a full implementation of the IMDP is conducted, taking the reader through the 
modeling phase, and then through the analysis phase to produce a set of linked meta- 
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models capable of describing the impacts of ESINQ factors and effects on operational 
effectiveness. Chapter V discusses the application of the meta-models developed in 
Chapter IV to improve operational and acquisitions support tools. The development of 
these tools will be discussed, and their applicability to support operation planning, TSE 
and analysis, and to support acquisitions decision making will be demonstrated. Chapter 
VI summarizes the dissertation’s key contributions and recommends future 
improvements to the methodology and tools described in this dissertation. Appendix A 
provides the MSSSM, which was executed to justify the selection of the M&S package 
used in this work. Appendix B supplements Chapter II by providing a more in-depth 
discussion regarding space threats and dependencies. Additionally, a Supplemental 
section is provided which gives a brief description of the two Microsoft Excel tools 
produce in this work, the IMDP and the IRCPAT, as well as how they can be requested. 
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II. A FOUNDATION 


If you know neither the enemy nor yourself, you will succumb in every 

battle. 

—Sun Tzu, The Art of War (quoted in McNeilly 2017) 

The purpose of this chapter is to provide more background regarding the 
motivation of this dissertation, focusing on the problem that it seeks to address and the 
context in which that problem resides. Specifically, this chapter will be broken into three 
sections, a foundational section, a framework section, and a keystone section. The 
foundation section, “The Three Knows,” will address U.S. space dependencies, adversary 
threats and capabilities, how the Army currently conducts operations and planning, as 
well as the expected operational impacts to U.S. operations from these threats. This 
section will frame the context in which the problem exists and provide Army specific 
motivations for this work. The framework section, “Mitigation Strategies and Gaps,” will 
discuss current and emerging Army mitigation strategies, focusing on doctrine, 
technologies, and Army modeling. This section will provide more general details 
regarding the linkages between the observed problem and the perceived organizational 
gaps, and while still addressing Army operations, will begin to highlight the more generic 
underlying issues within M&S practices. The keystone section, “M&S Community 
Modeling Gaps,” will close out the chapter with a discussion on what the author believes 
to be the underlying M&S gaps that have to this point contributed to the failure of the 
United States to adequately address adversary use of counter-space systems in both 
operational planning and acquisitions. This section will provide a more general 
discussion of the modeling gaps, and while decoupled from Army specific issues, as the 
keystone section, will tie the observed problems to gaps within the M&S community, 
highlighting potential areas for improvement. By organizing this chapter in this manner 
and exploring the problem and gaining an understanding of its context through the 
foundation, framework, and keystone sections, a conceptual bridge can be built to take us 
from the observed problem to a potential solution. 
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A. FOUNDATION: THE THREE KNOWS 

The U.S. military continues to place a greater emphasis on the importance of 
space as well as the advantages that space brings the warfighter. Over the past 10 years, 
the increasing emphasis on the positive value of space capabilities has been reflected in 
the growing occurrences of space verbiage in National Policy and Doctrine; where space 
support and capabilities have moved from brief acknowledgments of just a few sentences 
in the early 2000s, to the forefront of policy and doctrine in recent documents. 
Throughout these documents, the “Army has recognized its critical dependence on space 
capabilities across the whole force” (U.S. Army 2014a, 2). While the doctrine emphasizes 
the potential vulnerabilities of this critical dependency, it does not address any solutions, 
nor does it attempt to look at this vulnerability through the lens of the expected OE of the 
future, specifically operations in a D3SOE. This is a key point of this work; if the U.S. 
military does not consider this threat in the new and expected OE, we will produce 
mitigation strategies and systems that were designed for a different problem. This is a 
type I error and all too common in the DOD acquisitions community. 

This section is organized into three sub-sections, according to what Eung (2011) 
defines in his work as the “three Knows: Know yourself, know your enemy, and know 
the environment” (Ch. 1). The “Three Knows” parallels the quote from Sun Tzu that 
introduced the chapter and emphasizes the importance of understanding your own 
capabilities and limitations, as well as those of the adversary. This section will describe 
U.S. space dependencies, the major adversaries and their counter-space capabilities, as 
well as the environment in order to provide a better understanding of the problem. By 
understanding these aspects of the problem, as well as the overall OE in which it resides, 
decision makers can make more informed decisions when addressing threats and 
potential mitigation strategies when considering operations in a D3SOE. 

1. Know Yourself 

In the new AOC (2014b), the Army considers space-based capabilities as “first 
order capabilities the Army must possess to win in a complex world” (31). The term 
“must” is a fairly definitive statement. Couple this with other Army statements like the 
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Army must “assure uninterrupted access to critical communications and information links 
(satellite communications; position, navigation and timing; and intelligence, surveillance, 
and reconnaissance) when operating in a contested, congested, and competitive 
environment” (U.S. Army 2014b, 32), and we are quickly faced with a difficult problem. 
Simply put, in order for the Army to win in a complex world, it must have and maintain 
its access to space-based capabilities. Yet to do this effectively the Army must first 
embrace the idea of “knowing itself,” and strive to fully understand how it operates, the 
resources it uses, how it uses them, and all the intricacies that go with it. 

a. U.S. Space Dependencies 

The U.S. military achieves overmatch through a balanced combination of 
technology and force structure. In some cases, U.S. forces are actually much smaller than 
adversary forces, and sometimes even less capable when evaluated with traditional 
combat effectiveness metrics, yet through the expert application of technology in support 
of operations, the U.S. military is able to achieve overmatch. It is the application of 
technology that the United States arguably does better than any other country in the 
world, which allows the U.S. military to achieve operational superiority. But this comes 
with a cost, and to maintain overmatch it requires a substantial amount of space support. 
In fact, “the U.S. Army is one of the largest users of space-based capabilities within the 
DOD” (DOA 2009, 6), and likely one of the largest consumer in the world. This 
dependency on space support is at the crux of the problem, and “although our advanced 
space and cyber-space assets give us unparalleled advantages on the traditional 
battlefield, they also entail vulnerabilities” (DOD 2008b, 22). In order for the U.S. 
military to “Know Itself,” it must understanding these dependencies and vulnerabilities. 

According to the Union of Concerned Scientists (UCS) the United States is the 
largest operator of space systems in the world. As of August 2015, there were 1305 
operational satellites on orbit, of which the United States accounts for over 42%. While 
China and Russia, the next two biggest satellite operators are heavily invested in space as 
well, they only account for 11% and 10%, respectively. Of the 549 operational satellites 
currently operated by the United States, over half of them could be shown to have direct 
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operational relevance to military operations, and this does not even include the host of 
commercial satellites the U.S. currently uses. Thus, it is easy to see why many countries 
are confident in the assertion that the U.S. military is dependent on space-based systems. 
Using the satellite database compiled by the UCS (Union of Concerned Scientists 2015), 
the overall breakdown of U.S. military/govemment satellites (a sub-set of the 549 
operational satellites), to include commercial satellites, can be seen in Figure 7. 



Figure 7. U.S. Space Dependencies by Mission Area 


SATCOM accounts for the majority of military use of U.S. space systems, with 

60% of all satellites being focused on communications. The ability to rapidly disseminate 

information, SA, and execute Battle Management, Command and Control (BMC2), gives 

the U.S. military an unparalleled ability to quickly execute the decision making cycle—a 

significant advantage over adversaries who require more time to gather, process, and 

disseminate the information needed to make decisions. ISR is the second most prominent 

use of space systems by the United States, accounting for roughly 30% of all U.S. space 

systems. The ability to maintain up to date Situational Awareness (SA) regarding 
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adversary force distribution, disposition, strength, and location is often critical to decision 
making, and “supports the development of intelligence that supports mission success, and 
other actions that may influence the commander’s current and future operational 
decisions” (DOA 2014, 3-27). PNT accounts for another 5%, and while comparatively 
small when compared to SATCOM and ISR, due to its integration into the civilian sector 
has made it arguably the most critical. GPS allows decision makers to more accurately 
assess operations, the threat, and the environment, and allows them to make quicker and 
better informed decisions, which can drastically increase the operational tempo of the 
U.S. military. Together these three mission areas account for roughly 95% of all military 
satellite missions. Since operational requirements typically drive development, it does not 
take a huge leap of logic for our adversaries to surmise that U.S. dependency on space 
systems likely follows this same breakdown. The depth of space integration into 
operations highlights a potential dependency that is echoed in the Army Strategic 
Planning Guidance, which states that “the Army’s warfighting functions, weapons and 
battle systems are vitally dependent on space capabilities to achieve land dominance” 
(U.S. Army 2013a, 6). Thus, while the level of dependency on space is debatable, it is 
easy to see why adversary nations focus counter-space development in these areas. 

b. MDMP: How the Army Plans 

The Army has a clear and concise MDMP that has been refined over the course of 
many decades, producing a detailed “planning model that establishes procedures for 
analyzing a mission, developing, analyzing, and comparing courses of action against 
criteria of success and each other, selecting the optimum course of action, and producing 
a plan or order” (DOA 2005, 3-1). Army planners are extremely skilled at the execution 
of this process, and because of this skill, often take the lead in joint planning 
environments. While the MDMP is heavily used by the Army, it does have its limitations. 
First, it is resource consuming, both in time and in manpower, and therefore it is seldom 
used below the battalion level or in environments where the OE changes rapidly. Second, 
and more to the point of this research, it is only as good as the information that is 
provided. Figure 8 shows a graphical depiction of the MDMP. 
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Figure 8. The Role of the Commander and Staff in the MDMP. 

Source: DOA (2005, Fig. 3-1). 


The MDMP is a seven-step process that takes planners from receipt of the mission 
through orders production. The step that is of most concern to this work is step three, 
COA Development. During this step, opposing forces are analyzed and multiple COAs 
are developed that allow friendly forces “to generate overwhelming combat power at the 
decisive point to accomplish the mission at least cost” (DOA 2005, 3-115). A 
Commander will evaluate the staffs proposed COAs, and using his unique understanding 
of the OF, make a decision based on his/her acceptable level of risk and the mission 
objectives. During the first phase of this step, adversary forces are weighed against 
friendly forces, then threats and vulnerabilities are assessed, and finally combat powers 
are compared. This progression is often referred to as RCP analysis, and is defined as 
“the total means of destructive and/or disruptive force that a military unit/formation can 
apply against the opponent at a given time” (DOA 2005, 3-115). The tool that the Army 
routinely uses to conduct RCP analysis at the tactical level is the FRC, which can be seen 
in Figure 9. 
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RCP is a ratio of the Blue combat power over the red, and the higher the better. Leaders 
use this information to make decisions regarding force allocation. In this example, Blue is 
executing a hasty attack versus a hasty defense, where a RCP of at least 2.5:1 is 
recommended. Based on the assessed force strengths, blue has a RCP of 3.70:1, and thus, 
a decent advantage. 

Figure 9. Force Ratio Calculator. Source: Craig (1999). 


The tool was originally designed in 1999 by MAJ J. Craig during his CGSC 
course work, and allows planners to assess the RCP by selecting the type, quantity, and 
strength of the opposing forces, then selecting the missions of each. In this example, the 
friendly force has a 10.30 to 2.78 (3.7 to 1) RCP advantage, which is typically considered 
more than acceptable for an attack against a hastily prepared defense, which historically 
calls for at least a 2.5:1 advantage. While the FRC is a simple tool, which aggregates 
combat into the most simplistic of quantifiable measures, specifically system versus 
system without any accounting for additional sources of combat power, it is still the 
primary operational planning tool for accessing RCP of opposing forces. 
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2. Know Your Enemy 

While the first part of this section introduced U.S. dependencies on space, this 
second part will introduce the reader to the adversaries and the threat posed by their 
counter-space capabilities. When considering war, it is foolish to underestimate an 
adversary’s will or assume to know when or how they would employ their capabilities. 
Yet this is often the case with regard to U.S. space planners for three primary reasons. 
First, we assume that we can limit the scale of conflict to such an extent to deny an 
adversary’s use of specific capabilities. Second, we often focus on how an adversary will 
attack U.S. strengths and ignore our vulnerabilities. Third, we often assume that an 
adversary will act in a rational manner, and assume they have similar ways of thinking as 
us. Not only are these assumptions dangerous, they are illogical. As the cost and 
resources required for entry into space lowers, and as the proliferation of space 
technologies increases, the threats posed to the United States will continue to increase 
and we must accept that our “adversaries will likely attempt to harness the same 
advantages from space the U.S. currently enjoys” (U.S. Army 2014a, 5). In the OE of the 
future our dependencies and adversary capabilities will change so rapidly that it will be 
difficult to anticipate operational impacts. While there are actions the United States can 
take to mitigate these impacts, the dependencies on space can never be fully removed 
because space has become a critical enabler to both U.S. military and economic power. 
The truth is that space is no longer a safe haven for U.S. operations, and this fact must be 
acknowledged. As we move forward into the future, we can expect that space “will be a 
heavily contested environment, where the U.S. military will have to struggle to secure the 
information dominance that it simply presumed in the past would automatically obtain” 
(Tellis 2014, 5). With this understanding of the future OE it is possible to consider the 
potential impacts to combat effectiveness based on our knowledge regarding the 
adversary and its counter-space technologies. 

a. The Threat 

To avoid any misconceptions regarding the term adversaries, adversaries will be 
defined as those state and non-state actors who have developed “a range of counter-space 


30 



capabilities—with both reversible and permanent effects—designed to deny or degrade 
our ability to conduct military operations and to project power globally” (DOD 2014b, 7). 
To further delineate the term adversary, any country that the United States currently 
considers a defense ally, partner, or friendly party will not be considered. With this in 
mind, the list of potential adversaries can be reduced to just two, Russia and China. It is 
important to understand that for the purposes of this work, the actual adversary is for the 
most part irrelevant. The modeling being done in this work sets out to quantify impacts 
from operations in a D3SOE, and thus does not depend on whom we will face, but rather, 
it relies only on the potential counter-space capabilities that this adversary can bring to 
bear. Therefore, it is logical to choose the adversary with the greatest breadth of counter¬ 
space capabilities, and to make this determination let us briefly inspect the counter-space 
capabilities of Russia and China. 

Russia has a fairly significant counter-space capability, likely the third most 
capable following the U.S. and China. This capacity stems from the early years of the 
space program and through the end of the Cold War, when the USSR and the U.S. were 
in a long and protracted space race to achieve space superiority. Thus, Russia was able to 
procure a sizeable combination of counter-space systems and technologies, including 
everything from GPS and SATCOM jammers to Direct Ascent Anti-satellites (ASAT). 
Fortunately, a large portion of this technology has fallen into relative disrepair following 
the collapse of the USSR, and only recently, with the re-emergence of Russia as a world 
power, has Russia begun to increase its counter-space programs. But even with the re- 
emergence of Russia, and its renewed effort regarding counter-space capabilities, it will 
be years before its counter-space capabilities can rival that of the U.S. or China. 

Most space professionals consider China as a near-peer space faring nation, with a 
relatively new and modern counter-space program, which it has matured through a steady 
increase of resourcing over the last decade. While China’s counter-space R&D programs 
have less of a history when compared to the U.S. and Russian programs, China has been 
aggressively expanding its space and counter-space capabilities with little friction. In 
fact, over the last decade “China is developing significant anti-satellite capabilities, 
integrate cyber into all aspects of military operations, and developing sophisticated 
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missiles and air defenses as part of an effort to challenge United States’ ability to project 
power” (U.S. Army 2014b, 13). Couple this capability with regional aspirations to 
expand its influence into the Pacific, an area where the United States also has interest, 
and it is easy to see why many consider China as the nation most likely to challenge U.S. 
supremacy and thus, most likely to engage in escalating conflict that could lead to 
counter-space activities. Thus, due to the potential for conflict, the capabilities, and the 
willingness to employ such weapons, it is the author’s belief that the potential threat from 
Chinese counter-space capabilities far exceeds that of Russian capabilities, and therefore, 
open source estimations of Chinese capabilities will be used in this dissertation. 

b. Chinese Counter-Space Capabilities 

Over the last decade, China has developed a “multi-dimensional program to 
improve its capabilities to limit or prevent the use of space-based assets by adversaries 
during times of crisis or conflict” (DOD 2014a, 11), which provides some key insights 
into the internal thinking of the Chinese. This statement would suggest that the Chinese, 
who see the United States as heavily dependent on space, have decided that the most 
likely obstacle to their near-term and long-term national aspirations is the United States. 
And while many suggest that such a conflict would be detrimental to both economies, 
which are closely tied, China appears to be preparing for this nonetheless. As China 
identifies areas where advantage over the United States can be generated, it allocates 
resources to develop capabilities to leverage this advantage in order to generate combat 
power. What is most concerning about this is that China seems to have the initiative, and 
is doing it faster than the United States can respond. This paints a bleak outlook, and 
based on the expected growth of Chinese counter-space capabilities over the next 10 
years, it “ensures that almost every U.S. space component—the space systems in orbit, 
the links that control them and channel their data, and their associated ground facilities— 
will face grave perils” (Tellis 2014, 1). The growth of Chinese counter-space activities 
and programs closely parallels that of the perceived space dependencies described in 
Figure 7, and in nearly the same proportions. Thus, the Chinese are targeting their 
counter-space development activities to address the U.S. Space Force Enhancement 

Missions Areas, specifically SATCOM, ISR, and PNT. Since a large portion of the U.S. 

32 



military RCP is derived from the integration of teehnology and spaee based eapabilities, 
any sueeess of Chinese eounter-space systems in disrupting aeeess to spaee will have 
negative impaets on U.S. eombat operations. This may result in situations where the U.S. 
military is no longer able to generate enough eombat power to aehieve operational 
overmatch. The means by which China can accomplish this can be seen in Table 1. 


Table 1. U.S. Space Systems and Chinese Attack Options. 

Source: Tellis (2007, 46). 
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As shown in Table 1, the Chinese currently have feasible attack options for every 
one of the U.S. Space Force Enhancement Mission Areas. Depending on the number of 
counter-space systems used, China has the capability to significantly degrade the 
operational effectiveness of U.S. forces. This problem will only get worse as China 
continues to develop new and emerging technologies in the long term, like Co-Orbital 
ASATs (Co-ASAT) and directed energy weapons, which will increase the quantity and 
quality of China’s counter-space capabilities as well as the number of systems they can 
put at risk. Recognizing that any potential US-China conflict would likely take place in 
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the Western Pacific, the Chinese understand that they have what could be considered the 
home field advantage. Thus, they understand that the United States will need to rely 
heavily on space-based systems to overcome the “tyranny of distance” that can negatively 
impact battle management, command and control, and logistics of U.S. forces. In an anti¬ 
access, area denial environment which we expect, a D3SOE will serve to significantly 
delay a United States response into the Western Pacific, buying China time and space on 
the battlefield. Thus, as succinctly stated by Ashley Tellis (2014) in his brief to the House 
Armed Service Subcommittees on Strategic Forces and Seapower and Projection Forces, 
“the challenges confronting the U.S. military in regard to sustaining the information 
dominance it has traditionally enjoyed—in the face of current and prospective Chinese 
counterspace capabilities—will be enormous” (6). Unfortunately, there is currently no 
way to quantify the expected impacts from such challenges, at least not with respect to 
operational effectiveness, and thus no way to determine if the United States will be 
capable of maintaining operational overmatch in a D3SOE. This dissertation seeks to 
address this shortfall by developing a methodology that can be used to better inform 
decision support tools with regard to operations in a D3SOE. While this discussion just 
briefly discussed the counter-space capabilities of China, a more detailed description of 
the capabilities and impacts of these systems can be found in Appendix B. 

3. Know the Environment: Threat Impacts 

Most military planners know that understanding yourself as well as your 
adversaries is critical to making the most informed decisions. But many times the 
interaction of these two in the context of the OE is overlooked, or viewed as a secondary 
consideration, yet in many ways it is far more important. Without an understanding of the 
OE in which the two opposing “Knows”—’’Know Yourself’ and “Know Your 
Adversary”—interact, we risk overlooking a large number of potential effects that could 
impact operations. By considering the U.S. dependencies on space systems as outlined 
earlier in this chapter, as well as the counter-space capabilities currently available to 
potential adversaries within the framework of the OE we expect based on the newest 
AOC, we can obtain a better understanding of the OE and identify expected impacts. 
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a. 


The “New” Operational Environment 


For the first time in well over a decade, the Army has published a new AOC, 
which captures not only the lessons learned from the past 15 years of combat operations, 
but also addresses the new direction the Army must take to account for unknown 
adversaries and an uncertain future. This undertaking is fairly significant for the Army, 
yet complicated by the fact that “the Army is likewise reducing its end strength and 
seeking the ways and means to achieve efficiencies while preserving capability and 
capacity” (U.S. Army 2014a, 3). This AOC is significantly different than past documents 
for a few specific reasons. First, while past AOCs have addressed a specific threat, the 
new AOC outlines a future OF that has no specific adversary, no specific threat, no 
specific AOR, nor any specific tactical organization or mission. Second, the AOC 
captures for the first time the actual nature of modern warfare, which is highly chaotic 
and even unknowable. By embracing uncertainty rather than ignoring it, the Army hopes 
that it can better plan and prepare for the inevitability of operations in an unknown and 
uncertain future. Unfortunately, this more encompassing view of the OE introduces a 
fairly significant problem, because not only must the Army assure the nation that it is 
capable of defeating its adversaries, but it must do so without even knowing who, where, 
and with what it will be fighting. 

To face this problem, the Army has introduced the concept of Integrated 
Distributed Operations (IDO), which is one of the key tenets of future warfighting as 
described in the new AOC. IDO calls for the rapid creation and employment of heavily 
networked force packages and associated support services to quickly address threats. It is 
the hope of the Army that by maintaining a flexible, interoperable, smaller, and more 
agile force, that the Army can build tailored force packages that can apply overwhelming 
combat power at will to achieve decisive victories. Unfortunately, the IDO concept relies 
heavily on reach-back support for the generation of a large part of its combat power, and 
thus, “assured access to space capabilities will be critical to the success of the IDO 
element” (U.S. Army 2014a, 5). The dependency of IDO on space capabilities will affect 
the success of military operations in a D3SOE, yet the level will be impossible to 
anticipate because we are currently unable to quantify the impacts of space systems on 
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operational effectiveness, and thus, the impacts from a D3SOE. This problem is 
exacerbated by the fact that according to the new AOC, we no longer know the adversary 
or the systems they will use, which significantly limits our ability to prepare. The 
problem of planning for an uncertain future is even further complicated because our 
potential adversaries know exactly who they will fight, how they will do it, and the OE in 
which this conflict will take place. They will know our dependencies and vulnerabilities, 
and thus, our adversaries will continue to “invest in technologies to obtain a differential 
advantage and undermine U.S. ability to achieve overmatch” (U.S. Army 2014b, 11) to 
degrade the United States ability to project power. This puts us at a significant 
disadvantage when considering planning and preparation for future operations. 

b. Current Threat Assessment 

The ability of planners to more accurately plan for and anticipate impacts from 
adversary actions is a critical component of U.S. military superiority. Because of this, 
U.S. military operations typically deviate far less from what was planned for compared to 
our adversaries. Thus, our adversaries are routinely forced to be more reactive in order to 
recover from this deviation, which allows the U.S. military to gain and maintain the 
initiative, a significant operational advantage. Unfortunately this is not the case with 
regard to operations in a D3SOE, because while the U.S. military expects to be 
significantly impacted during operations in a D3SOE, we currently have no way to 
accurately anticipate these impacts. With respect to operations in a D3SOE, we are flying 
blind, forced to make assumptions of potential impacts to operations based on the most 
simplistic of heuristics. This, coupled with our greater dependency on space compared to 
any potential adversary, and we see a situation where adversaries can gain the initiative 
by simply forcing the U.S. military to operate in a D3SOE. To address potential 
mitigation plans, the United States must consider the requirements of space-based 
capabilities in the future OE and then assess the vulnerabilities of these systems 
compared to adversary capabilities. Eigure 10 shows the overall risk to U.S. operations 
due to the interaction of U.S. dependencies and adversary counter-space capabilities. 


36 




Figure 10. Estimated Risk to U.S. by PRC Counter-Space Capabilities. Source: 

Heginbotham et al. (2015, 251). 


Due to a mix of increased dependencies on space-based capabilities, as well as an 
increase in counter-space capabilities of China, the overall risk to the United States has 
steadily increased over the last 20 years. Of these areas, of most concern is the potential 
risk and impacts to communications, ISR, and PNT. As discussed earlier in this chapter, 
we know that the United States is heavily dependent on these capabilities, and that 
adversaries are focusing the majority of their efforts in these areas. But what is most 
noteworthy is that we do not see any decrease in risk. Thus, while we have long 
recognized the potential risks from operations in a D3SOE, the data suggests that we 
have been unsuccessful in mitigating the Chinese counter-space threat. To better 
understand the potential vulnerabilities associated with this threat we must also consider 
the potential growth of adversary counter-space capabilities over the near term. 

c. Threat Growth 

Based on the vision of the future OE as described in the new AOC, we know that 
the U.S. military can expect to face near-peer adversaries on their own turf, in situations 
where we will likely be significantly disadvantaged. We also know that to overcome 
these disadvantages the U.S. military will depend on technology and reach back 
capabilities to provide the deployed forces with the needed combat power to achieve 
operational overmatch. Additionally, we have been told that we must do this with a 
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smaller and more agile force, while maintaining current capabilities. Thus, with respect to 
the relative number of satellites, use, and perceived dependencies of the United States on 
space-based capabilities, it is not difficult to anticipate that these will all increase as well. 
If we expect the United States reliance on space systems to grow under the new AOC, we 
must also expect that potential adversaries will continue to grow and mature their 
counter-space capabilities. Thus, we will likely face a future OE where we are even more 
dependent on space-based capabilities to achieve U.S. national objectives, faced by 
adversaries who will be even more capable of negating those dependencies. Table 2 
provides a simplified assessment of the current counter-space capabilities of China 
compared to U.S. dependencies, as well as the estimated growth of both over the next 10 
years. This assessment, while based on simple assumptions, provides a rough estimate of 
future capabilities based on historic growth. 


Table 2. U.S. Space Systems versus Assumed PRC Capabilities. 

Adapted from Union of Concerned Scientists (2015). 



If current growth trends continue, the overall number of U.S. satellites as well as 
the number of adversary counter-space systems is expected to grow over the next decade. 
Unfortunately, the overall risk associated with this expected growth greatly favors the 
adversary, whose estimated growth of counter-space capabilities more than doubles the 
growth of U.S. systems. Currently, adversaries can degrade roughly 27% of regional U.S. 
space capabilities. Yet, within the next 10 years, it will likely be possible for advanced 

adversaries to degrade over 50% of U.S. space capabilities. While this is based on the 
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worst case scenario, where an adversary chooses to use all of its capabilities, which is 
unlikely, it serves to capture a key point...that mitigation through procurement is no 
longer a viable option. We can no longer presume to be protected by the relatively large 
number of systems we use, because eventually, the adversary will be capable of 
degrading all of them, at least regionally. We see this with ISR, where we anticipate that 
by 2025 that China will be capable of impacting all ISR systems simultaneously within 
its AOR. If we expand Figure 10 to account for the expected growth discussed in Table 2, 
we get an even bleaker outlook for future operations when considering Chinese counter¬ 
space capabilities as seen in Figure 11. 
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Figure 11. Expected Risk Growth to United States by PRC CS Capabilities. 

Adapted from Heginbotham et al. (2015, 251). 


As can be seen, the overall risk is anticipated to increase from moderate toward 
high over the next ten years. The majority of the growth in risk comes from the 
development of adversary counter-space capabilities to counter U.S. SATCOM, imagery, 
and PNT dependencies, and “the risk to most U.S. space functions appears to be growing 
faster than the U.S. ability or effort to mitigate them” (Heginbotham et al. 2015, 250). 
While the point at which the degradation of U.S. systems will begin to affect operational 
effectiveness is debatable, the impact is not. The U.S. military has become extremely 

proficient at collecting, processing, disseminating, and acting on information gained from 
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space; which has allowed us to gain and maintain a tactical and strategic military 
advantage. We have held this advantage for so long, fine tuning it to the point of near 
perfection, that any disruption to this system will likely induce cascading effects that we 
have not anticipated, trained, or prepared for. 

B. FRAMEWORK: MITIGATION STRATEGIES AND GAPS 

The threat from adversary use of counter-space weapons has been well 
documented. However, the preparation for such an event has not been executed nearly as 
well. And while the necessity to prepare for such an operational environment has been 
codified by the DOD (2012), stating that “the ability to compensate for loss of space 
capabilities will be integrated into joint and Military Department wargames, simulations, 
scenario development, experiments, and exercises” (4), the Army’s current mitigation 
strategies are attempting to address the problem in a piece-meal manner. The fragmented, 
un-synchronized, and “stove-piped” nature of the current Army space mitigation 
strategies has led to what can be termed as the “four referents of D3SOE mitigation.” The 
first of these is the Army combat operations referent, where activities focus on 
representing the ground combat environment, with impacts being assessed against 
measures of combat effectiveness. The second of these is the degraded space referent, 
which focuses on representing systems operating in and through space, as well as the 
threats they face, with impacts being assessed in terms of strategic and system level 
MOEs. The third is the mitigation strategies referent, which focuses on the representation 
of potential mitigation strategies that can address the threats identified from the degraded 
space environment, typically supporting engineering design M&S. The fourth is the 
acquisitions of space systems referent, which focuses on representing the operational 
environment as outlined in key systems engineering documents needed to acquire the 
mitigation strategies from the previous referent. Eigure 12 shows the authors depiction of 
these referents and their associated interactions. While this is a simple construct, and by 
no means an attempt to qualify the actual landscape of the four referents, it captures what 
the author believes to be a lack of a systematic mitigation strategy. 
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It is the author’s belief that the stove-piped nature of the Army’s D3SOE 
mitigation strategies has led to significant gaps in the ad hoc strategies that currently 
exist. From the author’s experience, the referents used to support Army combat modeling 
are quite robust, and adequately represent the majority of the OEs of interest to models. 
Eikewise, the degraded space referent is also fairly robust, and accurately represents the 
space environment and the interactions of systems within that environment. 
Unfortunately, the degraded space referent captures only a portion of the combat 
operations referent, and thus, if used for assessing impacts to operational effectiveness in 
the combat model, will be incomplete. The same can be said for the operations model, 
which fails to fully capture the referent of degraded space, and thus, cannot accurately 
model the impacts from space-based systems. This de-synchronization between the 
referents only grows as we look at the mitigation strategies developed in response to the 
understanding of the degraded space referent, which in turn drives acquisition decisions 
regarding potential mitigation strategies. This is a sequential process, where the 
understanding of the OE of the first referent is used as the input into the next referent and 
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so on. Current and emerging Army mitigation efforts typically take place at the overlaps 
between these referents, addressing the problem at each overlap individually, without the 
understanding of the full context. While the relative size of the referents in Figure 12 is 
purely subjective, meant to illustrate the general availability and saturation of these 
referents within the M&S community, it helps highlight the gaps between the four 
referents, which results in a de-synchronized mitigation strategy. It is the author’s 
assertion that these gaps are formed due to a lack of operational overlap between the four 
referents. And while individually these referents are more than adequate to represent the 
majority of OEs of interest, they are often incapable of accurately representing systems 
whose actual context resides in the zone of interaction between referents. 

By understanding current mitigation trends and the direction in which these 
emerging efforts are going with respect to the four referents of D3SOE mitigation, we 
can better highlight the areas where the United States is and is not actively advancing its 
efforts in dealing with the threat faced from adversary use of counter-space weapons. 
Because we know that “adversaries will continue to invest in technology to counter or 
evade U.S. strengths, resource reductions and insufficient force modernization place at 
risk the U.S. ability to overmatch its opponents” (U.S. Army 2014b, 41), the United 
States must take a more synchronized approach to mitigating the impacts from a D3SOE 
than in the past. The first step in this process is to address current mitigation efforts, and 
by doing so, it should be possible to fully identify and understand the capability gaps with 
regard to an adversary’s counter-space threats, as well as to help inform decisions 
regarding the allocation of resources. This section will identify areas that the United 
States has demonstrated a desire to advance its efforts to prepare, which usually take one 
of three forms. Doctrine, Technologies, or Modeling. 

1. Doctrine 

To better prepare for the potential operational activities, the Army relies heavily 
on the development of doctrine to codify how it will respond to a given number of 
potential situations. Typically, this doctrine takes two primary forms: TTPs and Policy. 
And while doctrine can never account for every possible problem or dictate a perfect 
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solution, it does help the Army prepare for likely situations. As highlighted in the 
National Security Space Policy (DOD 2011a), doctrine which has sufficient detail, to 
include means of implementation, can drastically improve the preparation for operations 
in a D3SOE. Figure 13 depicts where the Army is currently addressing mitigation 
strategies with regard to doctrine. 



Degraded 

Space 


Army Combat 
Operations 


Ivlitigation 

Strategie^ 


•^quisitionof 
Space Systems 


TTPs 


Policy 


Figure 13. Mitigation through Doctrine 


TTPs and Policy are applied at all three of the overlaps between the four referents 
of D3SOF mitigation, but are not mutuality supporting. While all overlaps are covered in 
one way or another, mitigation through doctrine fails to synchronize both TTPs and 
Policy at each of the three overlaps. TTPs tend to focus at the operational and tactical 
levels, giving direction to lower level units in preparation of combat operations, but this 
preparation cannot be limited to codification in national doctrine, it must “extend to the 
people and processes relying on space information, operating our space systems, and 
analyzing space-derived information” (DOD 2011a, 11). To do this effectively. Army 
forces must have trained and exercised in this environment, and the TTPs learned in that 


43 




process must be codified and documented in every unit’s standard operating procedures. 
While these TTPs cover only a small fraction of potential threats from operations in a 
D3SOE, they provide some basic capability for mitigation. Policy tends to focus at the 
strategic level, giving direction to higher level units during tasks like planning and 
acquisitions. The purpose of policy is to define the high level guidance of the United 
States, and provide a foundation for other efforts of the United States Government (USG) 
to prepare for operations in a D3SOE. While policy exists, the implementation for much 
of it has not kept up with changing technology, and thus, the United States has been slow 
to develop adequate means in which to deny adversary gains from use of its counter¬ 
space weapons, leaving the overall approach of mitigation vulnerable to exploitation. 

2. Technologies 

Technology is the key to implementing new and innovative mitigation strategies 
of the future. In the coming years, the Army will depend on these technologies “to help 
set the theater, surge capabilities for network and sensor assets, augment challenged 
space architectures, and reconstitute capabilities and forces after adversary actions have 
damaged or impaired space capabilities” (U.S. Army 2014a, 6). Through the creative 
application of technologies to address mitigation strategies identified in policy, the Army 
stands the best chance for preparing for operations in a D3SOE and maintaining the 
capability to fight and win in a complex world. While U.S. doctrine attempts to addresses 
the overall guidance that drives the development of mitigation strategies as well as some 
political maneuvering to reduce risk, doctrine in itself is not a complete solution. To truly 
implement a comprehensive mitigation strategy, technological solutions will always be 
needed. Eor the United States to be successful, it must be able to “defend friendly access 
to space capabilities, and ensure mission command access, by defeating or disrupting 
adversary attempts to deny, degrade, and destroy Army and joint access to space-based 
systems” (U.S. Army 2014a, 6). To do this effectively requires a combination of both 
Doctrinal and Technological strategies. Doctrinal strategies provide the logical “ways,” 
while technological strategies provide the physical “means” in which the mitigation 
strategies can be enacted. Eigure 14 shows where most technological mitigation strategies 
are currently addressed. 
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As shown in Figure 14, technologies are typically developed at the overlap 
between the mitigation strategies and acquisitions referents, though the technology itself 
will be applied at the overlap between the degraded space and combat operations referent 
once fielded. Thus, technological solutions developed and acquired to address mitigation 
strategies are done absent from direct input from the degraded space and combat 
operations referents, i.e., solutions are developed using a different understanding of the 
OE from the one in which the solution will operate. This is a type I error, and primarily 
due to the fragmented and unsynchronized manner in which the Army currently addresses 
mitigation strategies. Thus, while technology is developed to mitigate the risk and 
impacts from operations in a D3SOE, it is done so with an incomplete understanding of 
the OE, and absent an understanding of the full operational referent, will likely produce 
solutions that fail to meet their operational objectives. 
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3. Modeling 

The integration of M&S in the DOD has led to significant advancements over the 
past two decades, primarily in computing technologies that have drastically increased 
modeling capacity and applicability. Army M&S activities primarily support decision 
making in the R&D communities, but also support operations, planning, and training as 
well. The use of M&S has “become ubiquitous and indispensable to the Army as vital 
enablers of the Generating and Operating Forces” (U.S. Army 2017) and is a valuable 
tool for assessing the feasibility of mitigation strategies. To support a well-executed and 
informed MDMP and COA development, the Army uses a host of modeling tools and 
software packages that allow planners to better visualize and understand the OE. When 
aggregated, the outputs of these models provide a detailed and in-depth understanding of 
the interactions of the opposing forces within the OE. The majority of all combat systems 
and their individual contributions to RCP are well defined, including everything from a 
single soldier to a Battalion (BN) of tanks. These definitions often include variations to 
account for different sets of conditions like terrain and weather, which can all be 
accounted for during the MDMP. Unfortunately, as George Box (1987) famously said, 
“all models are wrong; the practical question is how wrong they have to be to not be 
useful” (74). The Army understands this dilemma, and rather than focusing on the 
production of answers to threats that are based on an inaccurate model, the Army instead 
seeks to use M&S to gain insights into the impacts from operations in a D3SOE. 

Within the Army space community there are four primary referents used to 
support M&S to gain the insight needed to support decision making; all of which have 
some impact on the Army’s ability to mitigate risk from operations in a D3SOE. The first 
is Army combat operations, where M&S can be used to support operations assessments 
and planning of mitigation strategies. The second is the modeling of degraded space, 
where M&S are used to model the impacts of a D3SOE on operations. The third is the 
modeling of potential mitigation strategies, where emerging systems and concepts can be 
tested for operational utility. And the fourth is M&S in support of acquisitions of space 
systems, where M&S tools are used to inform the acquisitions process of mitigation 


46 



capabilities. A depiction of how M&S is applied to the four referents of D3SOE 
mitigation can be seen in Figure 15. 



Figure 15. Mitigation through Modeling 


Army M&S activities are not focused at the overlaps between the four referents of 
D3SOF mitigation as we saw with doctrine and technology, but rather, because these 
referents are the foundation from which the individual M&S activities develop their 
models, they focus on capturing the specific referents of interest. This is a contributor to 
the inability of current practices to support the modeling of a D3SOE. To do so would 
require a shared understanding (referent) of the actual OF, and currently, M&S activities 
in each of these areas of D3SOE mitigation tend to be stove-piped, deriving their 
understanding of the OF from its own specific referent with little or no understanding of 
the other ones. The unsynchronized nature of current modeling practices is primarily 
caused by the issue of complexity in models. Because most models are developed for a 
specific reason to address a specific problem set, the resolution and detail of these models 
are typically maximized in the areas of interest, depending on the intended use. Thus, 
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areas of less interest, like the OE of the other referents, are modeled at a lower fidelity, if 
at all. The Army’s current use of models with respect to mitigation in each if the four 
referents of D3SOE mitigation will now be discussed. 

a. Army Combat Models 

The Army has a large contingent of combat models and can model ground combat 
operations to a high level of detail. Current combat models like COMBATXXI, OneSAE, 
and AWARS fill numerous operational support roles by providing key analysis across the 
operational domain where this information can help support decision making. 
COMBATXXI is the premier combat model currently in use by both the Army and 
Marine Corps and is likely the most complex and detailed model ever built by the Army. 
While this level of complexity makes it well suited for accurately capturing the 
environment for which it was designed (combat operations), it also limits the combat 
model’s ability to addresses other modeling referents, like operations in a D3SOE. It is 
important for model users to understand that due to the growing complexity of Army 
combat models, most models only consider the referents for which they were designed. 
With regard to the four referents of D3SOE mitigation, this means that combat models, 
nested in the combat operations referent, do not consider the impacts from the OE of the 
other referents. Thus, decisions made regarding mitigation strategies based on a combat 
model will not be accurate, because they largely fail to account for the actual OE, which 
includes a host of ESINQ effects, to include space-based effects. 

b. Modeling Operations in a D3SOE 

The modeling of operations in a D3SOE is often only a secondary consideration 
when compared to the modeling of combat operations. When coupled with the Army’s 
stove-piped manner of model development. Army models typically have a limited ability 
to account for operations in a D3SOE. Euckily, Air Eorce models can model space-based 
capabilities and the space environment fairly accurately. SEAS for example, is an Air 
Eorce combat model that is by far the most capable of all models investigated during the 
execution of the MSSSM with regard to modeling a D3SOE. Yet, similar to 
COMBATXXI, it also has some shortcomings due to its complexity. As with 
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COMBATXXI, SEAS looks to reduce model complexity by disregarding aspects of the 
OE to account for areas of interest. With regard to SEAS, an area overlooked was the 
ground combat segment, which is understandable when considering its purpose as an Air 
Eorce space model. Thus, as we saw with Army models, decisions made regarding 
mitigation strategies based on a space model will not be accurate, because they fail to 
account for all the interactions and dependencies between ground and space operations. 

c. Modeling of Mitigation Strategies 

To reduce the risk from the inherent inaccuracies of the information supporting 
decision makers, the modeling of potential mitigation strategies often focuses on 
standalone solutions for specific threats, thereby minimizing the chances of compounding 
inaccuracies. The relatively narrow scope of these targeted solutions creates an inefficient 
developmental environment which tends to ignore both positive and negative impacts 
from outside of the system boundary. This “stove-piped” approach to development tends 
to generate solutions driven by system level MOPs, constraints, and limitations rather the 
higher level MOEs and their impacts to metrics of operational effectiveness. These issues 
are compounded by the fact that both the Army and the Air Eorce address the 
development and modeling of potential mitigation strategies from different perspectives. 
The Army focuses on mitigation strategies that are tied to tactical MOEs, while the Air 
Eorce is concerned with higher level strategic MOEs. Unfortunately, the tactical and 
strategic MOEs are typically disjointed, and do not translate well from one to the other. 
While this current method of mitigation may work at times, it is by nature inefficient, and 
fails to take full advantage of the potential utility that could be gained by synchronizing a 
more formalized mitigation strategy. Thus, as we saw before, decisions made regarding 
mitigation strategies based on these constrained and incomplete referents will not be 
accurate, because they largely fail to account for interaction between the combat and 
degraded space OE. 

d. Modeling in Support of Acquisitions 

The Army’s current use of M&S in support of acquisitions is fairly well 
documented, yet with regard to the acquisitions of mitigations strategies, the process 
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becomes complicated. While the Department of the Army (DOA) (2011) has made it 
clear that acquisition program managers will use M&S to augment activities with the goal 
of increasing capabilities while minimizing cost and time, the sequential and widely 
distributed nature of the four referents of D3SOE mitigation makes it difficult to apply a 
synchronized strategy with regard to acquiring mitigation strategies. Modeling efforts in 
support of the four referents of D3SOE mitigation typically focus on assessing the 
contributions of alternative mitigation strategies in an effort to select the strategy or 
strategies that provide the most utility to the warfighter. Unfortunately, this assessment is 
typically based on the OE used by the acquisitions model, whose referent is only loosely 
based on combat operations in a D3SOE. Making accurate acquisitions decisions based 
on these inaccurate models is even further complicated by the breadth of acquisition 
activities in the DOD, which not only differ between each service but within them as 
well. Thus, as was seen before, due to the segregation of the four referents of D3SOE 
mitigation, their inability to capture the referents of the other OEs, as well as the differing 
objectives of the individual services, synergizing the acquisitions of accurate and well 
nested mitigation strategies is extremely difficult. 

4. Organizational Modeling Gaps (Army/DOD) 

While the current and emerging mitigation techniques described in this section, 
which included Doctrine, Technology, and Modeling, are steps in the right direction, the 
capabilities of the threat counter-space arsenal continue to increase. Whether current or 
emerging United States mitigation strategies will succeed in deterring aggression and 
protecting our space systems is yet to be seen. And as Heginbotham et al. (2015) 
described, the U.S.’s preparation for operations in a D3SOE “will depend on what 
investments the United States makes in space defense in the coming years and whether it 
can find ways to reduce its systems’ vulnerabilities” (257). Thus, the U.S. military must 
accelerate its efforts to anticipate adversary counter-space activities. While this will likely 
require a significant investment in resources as well as an increased focus from 
leadership, the DOD must be proactive in preparing for operations in a D3SOE. 
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Unfortunately, the Army has not thought it neeessary to put forth the resourees 
needed to fully investigate and quantify the eontributions of spaee-based eapabilities to 
RCP. Nor are there adequate modeling tools or software paekages available to support the 
level of analysis needed. While “spaee modeling-and-simulations will be needed 
inereasingly to support mission planning and rehearsal aetivities” (U.S. Army 2014a, 9), 
they do not eurrently exist in any form usable for space mission planners. While Army 
models do a fairly good job at representing the combat environment, they often aggregate 
and ignore secondary and higher-order interactions within the model to reduce 
complexity. Unfortunately, this aggregation typically takes place where many ESINQ 
systems and effects reside, and has resulted in an inability of current models “to represent 
the impact of new forms of command and control on combat outcomes because they are 
all based on physical models of attrition” (Cares 2004, 4). Thus, the Army finds itself in a 
situation where there are no M&S packages capable of accurately modeling the impacts 
of space-based capabilities on ground operations, the primary reasons for which fall into 
three broad categories. 

a. Desynchronized Mitigation Strategies 

The Army does not have the ability to accurately model dependencies on or the 
impacts from space-based systems, nor does it have a model to accurately compare and 
analyze different mitigation strategies. Even the Air Eorce, which can model mitigation 
of a D3SOE better through SEAS, fails to adequately tie its outputs to measures of 
operational effectiveness of the ground force. Thus, decisions and recommendation for 
the development of emerging mitigation strategies of both the Army and Air Eorce tend 
to have little-to-no traceability to actual measures of operational effectiveness, and fail to 
capture the actual OE. Due to the lack of traceability, the models used to inform decision 
makers regarding potential mitigation strategies typically only consider the requirements 
to acquire the strategy and are not well nested with either the D3SOE or the combat OE. 
Eigure 16 shows the U' order gaps within the four referents of D3SOE mitigation. 
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Figure 16. Gap Analysis: F' Order Systems 


As shown in Figure 16, the Army tends to focus its modeling efforts in support of 

D3SOE mitigation on just four referents of D3SOE mitigation. The lack of overlap 

between these referents has resulted in an inaccurate assessment of the OE from one 

referent being passed to the next referent. This work defines these as F‘ order gaps, 

which deal with the failure of each of the four primary referents of D3SOE mitigation to 

maintain traceability to the actual OE. These gaps can take two forms: complete gaps 

where no overlap exists, which are represented by red circles, and limited overlap gaps, 

which are denoted by yellow ovals. Starting with Army combat operations, one can see 

that there is little overlap with degraded space modeling, which is primarily an Air Eorce 

requirement. Thus, Army combat models fail to capture the space environment in the 

detail needed to represent space dependencies and threats, and Air Eorce degraded space 

models fail to capture the ground environment in the detail needed to assess operational 

impacts. Similar phenomenon can be seen at the other overlaps as well. The modeling of 

mitigation strategies receives only a partial understanding of the degraded space OE, with 

no direct consideration for the OE from combat operations. Eikewise, acquisition of 
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space systems also fails to directly account for the combat operations OE, with only a 
partial understanding of the mitigation strategies OE. This sequential approach results in 
each of the focal referents of D3SOE mitigation having its understanding of the OE 
informed through only a partial understanding of preceding referent. Thus, like all 
sequential systems, errors are aggregated and passed along, resulting in decisions that are 
not directly tied to either the combat environment or the D3SOE environment. 

The impact of a sequential process can be best illustrated by considering the four 
referents of D3SOE mitigation as children playing telephone. The first child, the combat 
model, informs the D3SOE model, but because of the limited overlap of the two, an 
incomplete understanding of the OE is passed. The D3SOE model then informs the 
modeling of mitigation strategies, but again, due to the limited overlap of the models, an 
incomplete understanding of the OE is again passed; only this time the error has been 
compounded. Eastly, the mitigation strategy model informs the acquisitions model and 
due to the limited overlap between the two, again provides an incomplete understanding 
of the OE. Therefore, the understanding of the OE that the acquisitions models use have 
little traceability to MOEs of the ground combat model, which according to proponents of 
Operational Effectiveness Modeling (OEM), is the primary measuring stick from which 
all decisions should be made. Thus, mitigated strategies are assessed using a skewed 
understanding of the OE, which fails to meet a key tenant of MBSE, which is to allow the 
systems’ impact to operational effectiveness be the primary drivers of design. 

To rectify this issue we need to develop systems whose purpose has traceability to 
combat effectiveness metrics, rather than the system-level MOPs that are typical of most 
sequential development approaches. To do so, we must scrap the sequential approach to 
problem solving and replace it with a more all-encompassing method for assessing gaps. 
While a sequential process can lead to a general increase in system level capability 
without consideration for MOEs, bigger is not always better: and MOPs like increased 
bandwidth and increased collection capability are not directly correlated to MOEs of 
improved combat effectiveness. Additionally, this type of approach is no longer 
justifiable in the current resource restrictive environment, which often requires metrics 
that can better quantify the return on investment. This goal is possible through a 
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capability-based decision and design methodology, which can consider the order gap 
as shown in Figure 17. 



Figure 17. Gap Analysis: Order Systems 


As shown in Figure 17, the 2"‘^ order gap captures all of the F' order gaps by using 
a broader and more encompassing methodology that can better account for the actual OF, 
which negates the flawed sequential process of F' order systems in lieu of a more robust 
and synchronized problem solving process. Simply put, 2"‘* order systems consider the 
OF of all four of the referents of D3SOF mitigation simultaneously as well as their 
higher level interactions with external systems and associated MOFs. By attacking the 
problem in this more holistic manner, we can more accurately account for the operational 
impacts from adversary use of counter-space weapons and therefore, more accurately 
predict the capabilities of emerging mitigation strategies. Unfortunately, the ability of 
decision makers to account for the 2‘’‘* order gap is significantly limited because current 
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methodologies and tools are not well suited for addressing the effects of external systems 
and non-quantifiable or intangible factors. 

b. Service Model Divergence 

While Army models like COMBATXXI, OneSAF, and AWARS do an excellent 
job of modeling ground combat operation, they capture only a fraction of the air and 
space referent, typically capturing just a few of the more rudimentary effects. Alternately, 
while Air Force models like SEAS, AFSIM, and SCT can model space and space-based 
capabilities in great detail, they do not model ground operations to the level of resolution 
needed to support the Army, nor do their MOEs translate to MOEs from the perspective 
of the Army. Thus, there is currently no M&S package that can accurately model the 
impacts of space-based capabilities on the ground force, because no one has attempted to 
build a cross-domain model whose referent accurately captures both domains. A 
graphical depiction of this divergence can be seen in Eigure 18. 
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Eigure 18. Ground and Space Modeling Divergence 
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As shown in Figure 18, both Army and Air Force models continue to increase in 
resolution/complexity, but primarily in the domain in which the models were designed. 
Thus, they continue to diverge from each other, increasing the desynchronization of the 
individual referents and moving further away from the model that is needed by Army 
space planners. In an attempt to resolve the disconnect, this dissertation will use the 
IMDP to capture the key elements of a D3SOE within a low resolution Army model to 
bridge the gap between Air Force and Army models. This proof of concept will provide 
decision makers a more robust and accurate model for operational and acquisitions 
decision support, while providing future model development efforts a framework for 
expanding models to account for cross domain effects. 

c. Modeling Domain Segregation 

Counter-space systems present a significant threat to the United States’ freedom 
of maneuver, and “to prevent enemy overmatch, the Army must develop new capabilities 
while anticipating enemy efforts to emulate or disrupt those capabilities” (U.S. Army 
2014b, 11). Yet the DOD does not currently have a dedicated cross domain M&S 
package that can effectively evaluate these new capabilities. Current DOD M&S methods 
still favor large and complex models that accurately model very specific domains, which 
can best be described in Figure 19. 
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Figure 19 is the author’s representation of the segregation of modeling within the 
DOD. Army models foeus on the ground domain, Air force models focus on the air and 
space domains, to include responsibilities for degraded space and acquisitions of space 
mitigation strategies, with neither model being able to do them both well. There are three 
major issues with acquiring mitigation strategies in this manner. 

1. Desynchronized referents of the services. Due to the limited overlaps of 
the service referents, any acquisitions process that produces a system to 
support the Army will do so with only a partial understanding of the Army 
ground combat referent. 

2. Differing domains of operations. Systems produced using primarily Air 
and Space operational domains will not perform as intended when 
assessed in the ground domain. 

3. MOE translation. Systems designed and acquired by the Air Force will 
have a different set of MOEs than the MOEs of interest by the Army; Air 
Force strategic MOEs do not translate well to Army operational MOEs. 

While models will continue to support the design, assessment, and acquisitions of 
systems, to include mitigation strategies, models do not seem to be evolving in parallel to 
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the changes we are seeing in policy and technology. When compared to both doctrine and 
technology, modeling has made the least progress with regard to preparations for 
operations in a D3SOE. And while models are flexible to change, this is only true to the 
point where it starts to impact its usability. There is a ceiling to model development, 
where no more complexity can be added without compromising some other aspects of the 
model. It is because of this ceiling that most models aggregate or ignore the effects of 
space-based capabilities on operations. Models are developed for a reason, and 
unfortunately, none of the current models used in the Army were designed to look at the 
impacts from operations in a D3SOE. While some models, like COMBATXXI, were 
designed to represent degraded BMC2, these aspects of the architecture were not 
exploited very well, though with the ongoing work of developers to represent behaviors, 
they could be in the future. To improve the acquisitions process of mitigation strategies 
and to produce systems that can provide more utility to the warfighter, a methodology is 
needed that can better link the Army and Air Eorce modeling referents. Until policy 
directs the development of a cross-domain model, one capable of modeling the impacts of 
external and context systems on ground combat operations, users will be forced to find 
unique and creative ways in which to use non-optimal models to assess ESINQ effects. 

(1) Modeling in Support of Operations 

The focus of Army planners is to support the overall COA development process 
by providing an assessment of the relative combat power. By definition, “Relative 
combat power analysis involves assessing tangible factors (such as, equipment, weapon 
systems, and units), and intangible factors (such as, morale and training levels)” (DOA 
2005, 3-30). Thus, for space planners, the focus should be on comparing friendly and 
enemy space assets, to include all potential sources of space-based force enhancement, 
like commercial assets. Unfortunately, the comparison of RCP of opposing space 
capabilities relies on the simplest of heuristics and qualitative assessments, giving space 
planners little quantifiable information to support decision making. While the Army 
MDMP and its supporting tools are well suited for assessing tangible factors, they do not 
account for intangible factors, and typically ignores space and counter-space intangibles 
altogether. The most common operational planning tool currently in use by the Army is 
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the FRC, which is based largely on the work of Ronald Misak (2001), who in his thesis 
work used the Army’s Consolidation of Forces Model (COFM) to attempt to quantify the 
contributions to operational effectiveness from specific forces. 

Unfortunately, as Zanella (2012) noted in his thesis work, the FRC has some 
significant shortcomings. First, the model has not been updated since 2001, when it was 
developed to focus on soviet-era tactics and threats, and thus, it is outdated and no longer 
represents current Army doctrine or modern OEs. Second, of the eight doctrinal sources 
of combat power (leadership, information, mission command, movement and maneuver, 
intelligence, fires, sustainment, and protection), the FRC is only capable of capturing two 
of them, fires and protection, and completely ignores the rest, likely due to being 
considered as ESINQ factors and effects. As noted by Zanella (2012), these “factors are 
traditionally ignored by modelers in equations because they are extremely difficult to 
quantify and replicate” (30). Third, combat power is additive, using a simple system 
versus system comparison, with no consideration for the synergistic effects of combined 
arms operations. Eourth, it does not account for force structures below the BN level. All 
in all, these failures typically lead to the consistent underestimation of friendly force 
combat power by the ERC, which routinely recommends a larger force than is actually 
needed. Thus, the Army’s current process ignores intangible or non-quantifiable factors 
during RCP analysis, when these intangible factors likely represent the majority of the 
sources of total combat power. Ignoring ESINQ factors like space-based capabilities is a 
fault of the MDMP, especially because they can often be just as important to the 
generation of combat power as the tangible factors. This concern is one of the key 
motivators of this dissertational research. The improvement to the MDMP through the 
quantifiable assessment of ESINQ capabilities will produce a more accurate 
representation of the OE and garner better and more informed decisions. 

(2) Modeling in Support of Acquisitions 

Similarly to modeling in support of operational planning, DOD Space 
Acquisitions typically uses some amount of M&S software to support decision making. 
Unfortunately, like operational planning, this support has been based on tools that do not 

accurately account for intangible factors like space and space-based capabilities. 
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Consequently, the Army Space community has had a poor history regarding the success 
of its space R&D programs, suggesting a potential linkage. It is the author’s assertion that 
the high failure rate of Army space R&D programs can be attributed to a lack of analysis 
during the early phases of the acquisitions process. Specifically, the author believes that 
this lack of early analysis is due to the failure to consider the potential contribution of 
new and emerging space capabilities to metrics of operational effectiveness of the force, 
imbedded in an accurate assessment of the OE. In the resource restricted environment of 
today, resources should be allocated to programs based on merit, specifically on how well 
a program contributes to the generation of RCP. 

Unfortunately, the ability of Army space acquisitions professionals to execute the 
type of analysis needed to quantify operational effectiveness is complicated by two 
primary factors. First, the Army space community considers its space programs as R&D 
and thus not subject to the same oversight and restrictions as programs of record. Without 
a more informed and formal approach to the allocation of resources to emerging 
programs, the Army Space community will likely continue its poor record of 
performance. Second, as we saw with operational support, there is currently no way to 
quantify the contributions of space-based capabilities to the warfighter. The inability to 
quantify these contributions is primarily due to the fact that there has been neither the 
need, the priority, nor a model capable of accurately modeling space enabled ground 
operations. One must ask, could space acquisitions professionals make better decisions 
regarding the allocation of resources if they had access to a tool that allowed them to 
analyze the tradespace of emerging space systems and compare each system’s ability to 
contribute to the RCP of the ground force? The author believes so, and a major intent of 
this dissertational work is to address this gap by providing Army acquisitions space 
professionals with a methodology and a set of tools that can better represent the 
contribution of U.S. space dependencies on operational effectiveness, as well as the 
impacts from adversary use of counter-space capabilities. 


60 



C. KEYSTONE: M&S COMMUNITY MODELING GAPS 

As we saw in the previous sections, the manner by which the Army mitigates 
threats from operations in D3SOE can be considered “stove piped.” By binning 
mitigation strategies into doctrine, technologies, and modeling, most strategies are 
developed to solve only a specific problem within a narrow view of the overall OE, 
without consideration for parallel and higher-level efforts. While this method can provide 
individual solutions for specific threats, it cannot fully capture the impacts from 
operations in a D3SOE because the models used cannot fully account for the operational 
dependencies of the ground force on ESINQ capabilities. The Army’s focus on high 
resolution combat models like COMBATXXI has complicated this problem even further 
because the requirements for these ground-centric models do not include dependencies on 
space-based capabilities. Without these dependencies, it is impossible to build an 
accurate representation of the OE, which has left most Army M&S packages less than 
optimal for addressing impacts to operational effectiveness in a D3SOE. Decisions made 
based on the output of limited models are inherently risky because they are based on 
incomplete knowledge. While the Army has been fairly successful in identifying and 
addressing the necessity to mitigate potential threats, it has been unsuccessful in 
mitigating them, primarily due to the turmoil that currently exists within the DOD M&S 
communities as well as a lack of a means to accurately quantify the impacts they may 
have on combat operations. This turmoil can best be described in Eigure 20. 
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Figure 20. Gaps and Modeling Chaos. 

Adapted from Davis and Blumenthal (1991, vi). 


According to Davis and Blumenthal (1991), multiple issues within M&S practices 
contribute to the chaos in combat modeling, which has left us with inadequate tools and 
processes to address current and emerging threats. As shown. Figure 20 captures all three 
of the observed organizational gaps discussed in the previous section, and highlights 
many potential gaps at the M&S community level. To link the observed Army 
operational modeling gaps to an overarching higher level M&S community modeling gap 
that could be filled to address the observed gaps at the organizational level, gap analysis 
was conducted, the traceability matrix for which can be seen in Figure 21. 
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Figure 21. Gap Traceability: Organizational to Community 


It is the author’s belief that this chaos observed in U.S. military organization 
modeling stems in part from two sources. The first of these sources, and the focus of this 
work, are the gaps at the M&S community level, specifically with regard to the in¬ 
flexibility of traditional MDPs to account for a more complete understanding of the OE in 
the development of the referent. By focusing on improving the underlying MDPs to 
account for ESINQ force multipliers during model development, it should be possible to 
produce better models, execute better OA, create better decision support tools, and thus, 
execute better and more informed decision making with regard to preparation for 
operations in a D3SOE. The second of the two sources, which was outside the scope of 
this work, is the belief that DOD senior leaders share some responsibility in enabling this 
failure. Models are developed based on requirements, and until a demand signal is 
sounded from the senior levels of the DOD to improve the capacity of combat models to 
address the gaps described in this work, only partial solutions will be possible. 
Unfortunately, with the threat growing faster than our capacity to mitigate, the United 
States is at a disadvantage and possibly unprepared to do what the AOC states it needs to 

“fight and win in a complex world.” Eor this issue to be resolved we must look for new 
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and innovative solutions that address not just the threats, but the higher order interaetions 
of the four referents of D3SOE mitigation, speoifieally at the M&S eommunity level. 

As described in Figure 20, there are gaps within current M&S practices that the 
author believes are hindering the ability of M&S practitioners to quantify the 
contributions and impacts from ESINQ effects on metrics of operational effectiveness. 
These gaps are further articulated by a 2010 JHU APE report which highlighted some 
significant gaps in M&S capabilities. These gaps included: what is modeled, referring to 
the aspects of the OE that are captured in the referent, and the ones that are ignored; 
limited modeling consensus, referring to the dissidence among M&S practitioners and the 
desynchronization of modeling across domains; M&S support to acquisitions, referring to 
the lack of any formali z ed process for using M&S in support of acquisitions; and the fact 
that “M&S developers lack understanding of modeling best practices, abstraction 
techniques, context dependencies, etc.” (JHU APE 2010, 1-2), referring to the inability 
of model developers to break away from the traditionally inflexible MDPs like those 
discussed in Section 2, and to embrace other modeling techniques that can better explore 
and capture the full context. These observations nest well with the gaps noted by Davis 
and Blumenthal in Figure 20, and help highlight the community level gaps that are 
currently limiting the ability of the U.S. military to prepare for operations in a D3SOE. 

Community gaps are more generalized than the organization gaps discussed in the 
previous section, and are focused on addressing the six primary high level gaps within the 
M&S community, and include: the inability to model ESINQ effects; the limitations of 
traditional MDPs; the compromise between error and complexity; the failure to accept the 
value of subjective assessments; the lack of an implicit model development process; and 
the desynchronization between design space and solution space saturation. Each of these 
gaps will be discussed in depth over the next six sections, and will support a better 
understanding of the context of the problem faced when attempting to bridge the gap 
between current modeling practices and the desired end state of this work. 
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1 . 


ESINQ Effects 


To capture a more holistic understanding of the OE in the referent, one capable of 
evaluating the dependeneies of systems on ESINQ effects, it “will require a fundamental 
shift from the current force paradigm based on expertise-centric missions and tasks, 
absent of tangible spaee-foree capabilities” (U.S. Army 2014a, 13). To support this shift 
to a paradigm that captures tangible ESINQ effeets, like spaee-based capabilities, a new 
methodology will need to be developed that can address the inability of current MDPs to 
aceount for the effects of external and context systems. By definition, external systems 
are those that are capable of impacting the system from across the boundary and can in 
return be affected by the system as well. Context systems on the other hand are defined as 
a “set of entities that can impact the system but cannot be impacted by the system” 
(Buede 2000, 50). Eigure 22 shows the context diagram of an operational model, with 
double-headed arrows representing the interactions of the model with external systems 
and single-headed arrows representing the interactions of the model with context systems. 
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Eigure 22. Modeling Context Diagram 


As shown in Eigure 22, the green area represents our system, which for this 
example is the combat model. The purpose of the model is to provide an accurate 
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representation of the aspects of the real world that are of interest to the modeler, which 
for this example may include the interactions of tanks, weapons, and artillery. Most 
agents in the system will be represented by hundreds of attributes that attempt to create a 
digital representation of the actual system within the model. While current models can 
account for the majority of the tangible interactions (green arrow) operating in the same 
domain, because of how the model was developed, they do not accurately account for 
intangible or non-quantifiable effects (orange/black arrows), especially across the domain 
boundaries. Thus, current models do not fully account for the complete OE in the 
development of the referent, even internal to the system boundary. 

The yellow area represents the external systems, which with respect to the combat 
model, account for the impacts of higher echelon forces and systems on the combat 
model (orange arrows). In reality, this interaction would include the physical 
contributions to the model from all domains. However, in modeling these higher systems 
and effects are typically ignored, and the majority of their contributions to the model are 
either lost or aggregated within the modeled agents. This deficiency is primarily due to 
the fact that the contribution of many of these external systems to the current model can 
only be partially quantified, because many are behavioral impacts rather than physical 
impacts. For example, while the effects from higher echelon artillery and air strikes can 
be quantified in a combat model, effects from deep strike, strategic intelligence, BMC2, 
communications, logistics, and even political pressure are more difficult to capture 
because their observable impacts cannot be accounted for in physics based models. Thus, 
the ability to accurately model the OE is further reduced because current models cannot 
fully capture the OE of the external systems. The accuracy of the model is further 
reduced by the translation errors associated with modeling cross-domain effects in terms 
of the model’s domain attributes. For example, consider modeling the impacts of cyber¬ 
attacks in terms of a ground combat model. To do so would be difficult because the 
attributes available within the combat model (a physics based entity model) are based on 
the ground domain, which typically do not include measures that can accurately define 
the effects of cyber operations, who’s impacts do not directly affect the capabilities of the 
system. In reality, these effects would likely be observed as a combination of behavioral 
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effects, like slower decision making cycles by leaders leading to negative impacts within 
the model, as well as reduced SA, which would be observed as an increase in location/ 
targeting errors, also leading to negative impacts to the outcomes of the model. Yet, 
physics based models cannot adequately, if at all, address behaviors, which precludes 
applying such effects directly to the modeled systems, because in reality, their physics 
should remain unchanged. To overcome this issue, the impacts of these effects must be 
applied to the systems indirectly, and thus, a significant portion of the resolution needed 
to model cyber effects is lost due to the inflexibility of the ground combat model to 
account for systems outside the domain in which it was designed. 

The red area described in Figure 22 represents the context system, which in the 
example of the combat model, represents the impacts of strategic forces like space and 
other ESINQ effects. As it is with external systems, this interaction is real, but the distinct 
difference is that the tangible contributions to the model from context systems cannot be 
measured, or more accurately, “is not measured.” Thus, context systems are often 
considered intangible and mostly ignored in models. While it is possible to incorporate 
context systems into the actual model, it would induce a significant amount of 
complexity, again due to the difficulty in translating behavioral impacts to observable 
impacts within physics based entity models. Thus, modeling efforts that attempt to 
capture some of the effects of external and context systems are only partially successful, 
as seen in Figure 22 (orange/black arrows), primarily due to three main issues. First, 
accuracy is lost in the representation of external and context systems and their tangible 
effects when integrated across the system boundary. Second, the model cannot account 
for seemingly intangible or non-quantifiable contributions, especially ones which 
primarily impact behaviors. Third, much is lost in translation when attempting to model 
cross-domain effects of external and context systems, which operate in domains other 
than the domain of the model and are often difficult to accurately represent in physics 
base models that typically lack the capability to model behavioral impacts. 

These issues have created a situation in which model developers are unable to 
accurately account for ESINQ factors and effects, limiting them to just three methods to 
try to capture the effects of systems external the system boundary. In the first method. 
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developers can choose to ignore all interactions from outside of the system boundary. 
Depending on the intent and the purpose of the modeling effort, this may be a valid 
method for addressing certain objectives and goals, though careful consideration on the 
impacts from failing to account for the actual OE will need to be considered. While this 
seems like a faulty approach, it is common in the M&S community because it induces no 
increase in complexity, but it does fail to increase the accuracy of the model. In the 
second method, developers can add the functionality of the external forces and systems 
into the model by expanding the system boundary to encompass the external systems of 
interest across all domains. While the new expanded model would capture the tangible 
impacts of the external systems, as well as an increased representation of the OE, it 
would require a significant increase in complexity of the model, and require a substantial 
investment of resources to implement. Because of this complexity, this method is often 
limited to M&S developers with significant resources, like military and government 
agencies. In the third method, developers can select a specific external system or effect in 
which they are interested in, and insert these into the model as agents. While this would 
increase the model’s capacity to account for the effects of these systems, it would do so 
through only a moderate increase in complexity due to the addition of agents and their 
associated attributes. Unfortunately, the ability of these agents to capture the actual 
effects of the systems which they aim to represent would be limited to just the tangible 
effects and level of resolution the model attributes allow. Simply put, models that attempt 
to represent systems from domains other than the ground domain, or systems that have 
seemingly intangible/non-quantifiable effects, or impacts to behaviors, will find it 
difficult to adequately represent their effects when they are limited to the agent attributes 
available in the physics based ground combat model. Yet, even with the inability to 
accurately address cross-domain systems, this is the method most model developers use. 

Unfortunately, all three of these methods have significant disadvantages in terms 
of modeling. The first method ignores everything outside of the system boundary 
resulting in a less complex but less accurate model. The second method is far more 
accurate and captures a larger representation of the OE, but is considerably more complex 
and difficult to achieve, and can only partially represent these effects, especially impacts 
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to behaviors. And the third method is only marginally better than the first, gaining 
aeeuraey by eapturing some of the tangible effeets of external and eontext systems at the 
eost of inereased eomplexity, but it fails to aoeount for the seemingly intangible and non- 
quantifiable effeets that impaet behaviors. Luekily, this work supports a fourth option, an 
improved method for aceounting for ESINQ effects. 

While the effects of ESINQ systems like space and counter-space systems are 
hard to quantify, especially when these impacts can be classified as behavioral, we should 
be able to represent the effects they produce through the creative manipulation of 
surrogate factors. If we can capture just the significant effects of ESINQ systems on the 
entity model, to include impacts to behaviors, in a manner that can be translated to the 
model even after considering the significant aggregation and simplification of the effects 
due to the cross domain translation effects, we should be capable of providing a much 
better representation of the actual OE. Using this improved understanding of the OE, 
developers should be able to quantify and translate the effects of these ESINQ factors and 
effects into the operational model. While this IMDP would result in only a partial 
improvement of the model, it would for the first time address external, context, and 
ESINQ systems in a deliberate fashion, to include impacts to behaviors which are 
typically not addressed in modern physics based models. The model developers can then 
insert these effects into the model as modifications or adjustments to a few specifically 
identified agent attributes (surrogates) that already exist in the model. While these 
modifications would not increase model complexity, they would yield an increase in 
accuracy of the model by capturing effects across system boundaries and domains to 
include the tangible effects of the ESINQ effects that were previously ignored. While this 
method would require an upfront commitment of resources, such complexity would not 
be translated to the model. Thus, for models used frequently, the initial investment of 
resources would be more than justified. 

2. Traditional MDPs 

While a plethora of MDPs are available for use, most of these MDPs provide only 
a simplified framework for executing model development. Couple these weakly defined 
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MDPs with the fact that most M&S users tend to distill these processes down even 
further, into an even more generalized framework which leads users through the 
definition, development, and analysis of a model. The issue with this is best captured in 
the following statement: “Although the importance and use of modeling and simulation 
(M&S) tools (models, simulations, and utilities) is expanding across the Department of 
Defense (DOD), relatively few persons have a good grasp of the process and principles 
that should be followed when developing such tools” (JHU APL 2010, ES-1). Thus, not 
only are traditional MDPs inflexible and poorly defined, but most users fail to even 
adhere to the overly simplified rules established in these MDPs. And while there are 
various schools of through within the M&S community regarding the importance of the 
model definition, specifically the development of the referent, Ilachinski (2004) for 
example, this work aligns with the more common of these schools of thought, that the 
referents establishes the foundational understanding necessary for the development of the 
model. To better understand the shortcomings of such traditional MDPs, let us begin by 
briefly discussing the history of MDPs, as well as exploring a few of the more prominent 
MDPs current in use within the community. 

Models have been in use for 100s of years, likely longer, yet until recently these 
models were for the most part either conceptual or physical models (scaled analogs of 
something more complex that were often used as tools in support of gaining 
understanding). Following the advent of the computer, the definition of models was 
expanded in the 1960s to include computer-based models, which included the computer 
program as a legitimate medium for representing a model. With the technological 
advancements that computers facilitated, specifically in the rapid evolution of computing 
power, memory, and storage, computers drastically increased the capabilities of model 
developers to capture a larger and more accurate representation of the systems being 
modeled. With the rapidly expanding potential of computer-based models it became 
apparent that a more formali z ed process was needed to deal with the nearly exponential 
increase in model complexity. Enter the traditional MDP. Being rooted in the already 
existing systems development processes and functional models of the time, the MDP 
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sought to codify a more detailed process for developing computer-based models. Figure 
23 shows what the author considers the four most commonly used traditional MDPs. 
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Adapted from DOD (2008a, 8-18), JHU APL (2010, ES-2), Law (2015, 67), Sargent 
(2001, 109). 


Figure 23. Four Common Traditional MDPs. 


Each of these four MPDs was developed to attempt to codify the process for 
developing computer-based models, yet each author did so with a specific focus and end- 
state in mind. Thus, there is significant variation among traditional MDPs, and to 
highlight the commonalities and differences, each MDP will be discussed in turn, starting 
with arguable the most widely accepted MDP, the one developed by Law (2013). 

In his book. Law (2013) provides a basic, yet well-articulated MDP which he 
refers to as “steps in a sound simulation study” (66). While he devotes less than four 
pages to this topic, it is enough to support the development of a general model 
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development framework. Law does this by breaking his process down into 10 steps, three 
steps devoted to model definition, five steps devoted to model development, and two 
steps devoted to model analysis. He then provides a short general description of these 
steps, but with little detail regarding how they should be accomplished. Yet this is 
understandable. Law was attempting to provide a MDP that retained enough flexibility to 
be useful for most potential users, which required a more generalized process. When 
considering the nearly infinite uses for models, any MDP that was too specific would 
significantly reduce its potential usability. Yet, even after considering his reasoning, the 
MDP developed by Law seems to be lacking the detail necessary to conduct what he calls 
“a sound simulation study,” specifically with regard to model definition. 

The next MDP which will be discussed is the one developed by Sargent (2001), 
which he refers to as a “simplified version of the model development process” (107), 
which he later expands to include the application of model verification and validation as 
seen in Figure 23. Sargent breaks his process down into three steps, two steps devoted to 
model definition, and the third step devoted to model development, yet as with Law and 
Ilachinski, he does not provide any description or detail regarding how they should be 
accomplished. While Sargent does put more emphasis on the importance of the model 
definition step, his focus on producing a validated model overshadows the importance of 
the model definition step, specifically with regard to the development of an accurate 
referent. This highlights another issue with traditional MDPs, where the end state of 
many MDPs is seen in the development of a validated and accredited model rather than 
the development of the most accurate model possible. 

The third MDP which will be discussed is the one developed by the DOD in its 
Modeling and Simulation Body of Knowledge (2008). In this work, the Modeling and 
Simulation Coordination Office attempts to consolidate and standardize M&S knowledge 
for DOD users, and in doing so, codify a process of model development. While this work 
does a slightly better job at describing the steps of model development, it is done in a 
desynchronized manner, providing users with a loosely organized list of knowledge area 
concepts and associated descriptions, but with little-to-no traceability between the 
concepts or detail regarding how to execute them. Thus, users are forced to determine 
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which areas are pertinent to their work, how these areas should be executed, and then to 
link these concepts into a framework for implementation. While this approach may be 
useful in codifying a body of knowledge, it is not very useful as an executable process, 
failing to define the linear/iterative process necessary to develop models as well as the 
importance of establishing an accurate referent. Figure 23 highlights this issue, where the 
author was forced to cut and paste specific knowledge areas from throughout the lengthy 
document into a structure that resembles an executable MDP. 

The last MDP which will be discussed is the one developed by JHU APL (2010). 
In this work, titled “Best Practices for the Development of Models and Simulations,” 
JHU APL took a unique approach to establishing its MDP, specifically through the 
distillation of existing SE practices, which were then each “assessed to identify its 
applicability to the M&S domain, along with its relative strengths and weaknesses. The 
results of these assessments were synthesized into a new SE Eramework” (ES-1) for 
model development. JHU APE breaks his process down into five phases, not including 
the project management practices. Of these phases, two phases are devoted to model 
definition, and three phases are devoted to model development. As with Sargent, JHU 
APE puts more emphasis on the model definition step, which is very much in line with 
traditional SE practices, and thus, does a better job at capturing the importance of model 
definition than most other MDPs. Yet, differing from the other MDPs investigated, JHU 
APE put a significant amount of effort into codifying the details of their MDP, which can 
be found in their Best practices Definitions (JHU APE 2010, B-1-13). Additionally, and 
of significant impact to this work, JHU APE identified a general lack of detail in other 
processes as a significant gap within the M&S community, stating that: 

The most notable observation about this effort is that, although there have 
now been decades of focus on engineering processes and process 
improvement, much of it has been focused on systems and software in 
general, not on models and simulations specifically, and much of it at the 
macro level, rarely daring to drill down to the level of individual best 
practices. The study team was surprised by the lack of detailed best 
practices for the development of models and simulations in the literature. 

JHU APE (2010) 
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With a review of four of the more common MDPs currently in use, let us now 
explore these MDPs more generally by considering just the two general steps of most 
traditional MDPs, model definition and model development. Although analysis is 
typically the third step in most MDPs, it is not discussed here because its process is 
unaffected by the implementation of the IMDP. The generalized framework, overlaid 
with four examples of traditional MDPs can be seen in Figure 24. 
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activities. 


_ Model Development _ 

• Primary focus (70%). 

• A general description of activities with 
moderate detail. 

• MDPs are fairly well understood. 


_ Analysis _ 

• This is the goal (10%). 

• Well understood aspect of traditional 
MDPs, little need for additional 
insight. 


Adapted from DOD (2008a, 8-18), JHU APL (2010, ES-2), Law (2015, 67), Sargent 
(2001, 109). 


Figure 24. Steps of a Traditional MDP. 


Model definition includes the steps of identifying the problem, formulating a plan 
of action for the simulation study, and gathering all necessary information needed to 
inform model development, to include the development of the referent. Model 
development is concerned with the actual development of the model, to include test, 
evaluation, execution, and experimentation. This is the primary focus of most traditional 
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MDPs, and thus, receives the majority of the resources. Because of this focus, these steps 
are fairly well understood and articulated in modem writings. Most commercial M&S 
packages provide detailed model development processes, and almost all users of M&S 
are well versed in their own adaptation of these processes. Unfortunately, as described in 
Figure 25, there are some significant gaps with traditional MDPs. 
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Model Definition 

• Relatively small focus on this 
foundational step. 

• Very little detail, and doesn't describe 
"how" to execute model definition. 

• Un-formalized and weakly defined. 

• Typically focuses on known system 
specific data, ignores the rest. 

• Discourages subjective and implicit 
methods for addressing ESINQfactors. 

• Assumes users understand the context 
and will establish an accurate referent. 

_ Model Development _ 

• Based on the referent developed 
during model definition. 

• Inflexible MDPs tend to take a purely 
explicit model development approach 
while ignoring anything that may 
induce subjectivity into the study. 

• Assumes users will use appropriate 
M&S tools. 


Adapted from DOD (2008a, 8-18), JHU APL (2010, ES-2), Law (2015, 67), Sargent 
(2001, 109). 


Figure 25. Gaps of a Traditional MDP. 


As shown in Figure 25, most traditional MDPs underwhelm the model definition 
step, devoting a relatively small portion of the overall resource budget to this critical step, 
often assuming the user will ensure an adequate understanding of the OF is instantiated in 
the referent prior to model development. Thus, this step is for the most part un¬ 
formalized, weakly defined, and lacking any specific detail regarding how to conduct 
model definition, offering just a simple framework or best practices for users. This was 
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highlighted during the November 2015 dissertation defense of Sam Sok, who noted that 
with regard to MDPs, “All processes start by attempting to define the system. None of the 
processes explains how to define the system” (14). This lack of detail may result in users 
who hurry through this step and often limits the type of input data to quantifiable sources, 
which significantly limits the capacity of current MDPs to account for ESINQ effects. 
This incomplete understanding of the OE is then passed on to the model development 
step, which relies on the quality of the model definition step, which as described, is 
lacking in detail. The problem is further complicated by the fact that most traditional 
MDPs take a purely explicit model development approach, directing users to avoid hard- 
to-quantify input sources for fear of injecting subjectivity into the study. Thus, most 
traditional MDPs are ill-equipped to model systems or effects that can be considered 
ESINQ, resulting in the majority of ESINQ factors and effects being ignored. 

The fundamental issue with the models produced using traditional MDPs is that 
the underlying assumptions and methods for gathering data during the model definition 
steps, specifically the development of the referent, are overly simplified. Thus, the 
models developed will fail to represent the OE and the systems they were intended to 
model, resulting in analysis based on an incomplete and more inaccurate model. The 
primary reason for the inability of traditional MDPs to capture an accurate assessment of 
the OE during model definition is their failure to recognize the existence of more than 
two sources of combat power in developing the referent. Eigure 26 is a graphical 
representation of the author’s interpretation of the sources of combat power captured in 
the referents of traditional MDPs. 
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External Contributors 



Figure 26. Traditional MDP Referent Contributors 

As noted by MacCalman et al. (2016), traditional MDPs focus on just two 

contributors for the generation of the referent, where “the input parameters to the 

operational simulation are typically classified as one of two types” (2). These types 

include internal contributions, which capture key known system contributions, and 

external contributions, which bound unknown contextual contributions (typically 

distractors or degraders) of interest. The intent of the model (the green oval) is to capture 

enough key aspects of the system in the context of the physical world to provide an 

approximation of the system of interest, to include some of the interactions with the OE. 

Thus, the majority of the modeled combat power is derived internal to the system, with 

only minimal accounting of other contextual factors. Unfortunately, this approach ignores 

a sizeable portion of the actual contributions of both the system and the environment (the 

blue cloud), which can often contain a significant portion of the total combat power of the 

things being modeled. If the ignored sources are significant, then the models will 

underestimate a system’s total contribution to operational effectiveness, an artifact that is 
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often seen in modern combat modeling. Of interest to this work is that the areas typically 
being ignored are often where external dependencies and ESINQ factors and effects 
reside. It is the belief of this author that there are more than two contributions that should 
be considered in the development of a model’s referent. The first expansion of Figure 26 
addresses the inability of traditional MDPs to recognize and account for the dependencies 
on external systems for the generation of internal combat power. This expanded view of 
modeling can be seen in Figure 27. 



Figure 27. Expanded MDP Referent Contributors (External Dependencies) 

As shown in Figure 27, by separating system internal sources of combat power 
into two areas—a system’s internal contributions and a system’s internal contributions 
that are externally dependent—it is now possible to decouple any dependencies the 
system has on external resources to generate internal combat power. Because most 
traditional MDPs either ignore these contributions or aggregate them with internal 
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contributions, most MDPs cannot account for the impact that external dependencies have 
on internal metrics of operational effectiveness. By expanding the sources of combat 
power from two to three, it is now possible to account for these sources separately, 
allowing us to capture a more accurate representation of the OE in the referent. In turn, 
users can degrade the portion of the system’s combat power that is dependent on external 
support, which facilitates the quantification of the operational impacts due to a D3SOE, a 
key outcome of this work. The second expansion of Eigure 26 addresses the inability of 
traditional MDPs to delineate between unknown and known external contributions to 
combat power. This expanded view of modeling can be seen in Eigure 28. 



Eigure 28. MDP Contributors Expanded for ESINQ Eactors 


By delineating between contextual factors and ESINQ factors, we highlight a 
fourth area of potential contributions of combat power that is currently unaccounted for 
within the referents of traditional MDPs. The key difference between the two is that 
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contextual factors are by definition unknown and/or uncontrollable, but ESINQ factors 
are not, they are both known and controllable, though they are often extremely difficult to 
quantify. By implementing an improved MDP capable of iterative implicit model 
development, it should be possible to expand the model to include all four of these 
potential sources of combat power within the referent and loosely quantify or “bound” 
the contributions from both externally dependent and ESINQ sources. While the majority 
of the modeled attributes are still derived internal to the system, the model can now 
account for all four potential sources of input, and thus, provides a more accurate 
depiction of the actual OE. 

3. Error versus Complexity 

A significant limiter to accurately representing the OE is the problem of 
complexity. In modeling, accuracy begets complexity, and complexity negates modeling. 
Simply speaking, there is a constant give and take between model accuracy and model 
error, and you cannot affect one without the other. This conflict was best described by 
Eeinweber (1979) in his work “Models, Complexity, and Error,” as seen in Eigure 29. 


ERROR 


Eigure 29. Error versus Complexity. Source Eeinweber (1979, 11). 

Error of Specification (Cs) decreases as you add more detail (complexity) to the 
model due to the inclusion of more information regarding the actual OE. Alternatively, 
Error of Measurement (Cm) increases as you add more information due to the 
compounding of inherent measurement error. Thus, there is a point of minimum error in 
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modeling that limits the amount of eomplexity a user would want to inelude. Any 
additional eomplexity added past this point would induee an inerease in overall error and 
thus, may not be worth the effort. What is needed is a means to inerease a model’s 
eomplexity (accuraey at representing the aetual OE) without the reciprocal increase in 
total error. One way this may be possible is through the creative use of implicit modeling. 
What if we could extend Ci from Figure 29 to the right, allowing us to gain more 
accuracy before total error begins to increase? An example can be seen in Figure 30. 



Decrease error 
from ej to eT 2 


ERROR 


Increase accuracy 
from Cl to C 2 


Figure 30. Error versus Complexity. Adapted from Feinweber (1979, 11). 

By using implicit modeling techniques to capture the effects of ESINQ factors 
and effects on previously modeled attributes, it should be possible to increase model 
accuracy without increasing complexity, i.e., to reduce the slopes of both the Cs and Cm 
curves. Because the modeled agents can implicitly account for more than themselves, 
representing a larger portion of the OE, they induce less error of specification. Fikewise, 
because the modifications of the surrogate factors within the models were calibrated, the 
error in measurement is also reduced. Thus, it allows for a more complex model before 
reaching the point of minimal error, which could be lower than the original. 

4. The Value of Subjective Assessments 

Another limitation of current MDPs is the common belief that subjective 


assessments are something to be minimized or avoided. This unfortunate trend within the 

M&S community has resulted in most MDPs pushing users toward more explicit studies 
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using primarily quantifiable data sources. While subjective assessments can indeed lack 
specific details, and do often rely on human assessments, they are not without merit. In 
fact, depending on the intent and implementation of the study, many subjective 
assessments can be extremely useful and provide analysts with unique and valuable 
insights that would have otherwise been ignored. Even if the study is somewhat 
subjective, if the study improves analysis and the decisions that result from that analysis, 
then it adds value. The benefit of coupling both explicit and implicit modeling techniques 
is best demonstrated by considering military decision making. 

In the Army, generations of leaders have been matured with a firm understanding 
of the balance between the art and science of decision making. The science of decision 
making is typically backed with quantifiable facts and data, often sourced through 
intelligence and other trusted collection resources. Yet military leaders understand that 
there is a limit to what can be knowable and that for everything that is known, there is 
often more that is not. The art of decision making focuses on addressing these unknowns. 
It is by nature more subjective and relies heavily on the leader’s own knowledge and 
experience, as well as assessments of his/her staff. Military leaders understand that they 
will not always have the best picture, but through a disciplined process like the MDMP, 
they can successfully plan and execute operations, even in the face of uncertainty. Good 
leaders are the ones that can merge both the art and science of decision making and use 
them to achieve better operational outcomes for their forces. 

To achieve the balance between the explicit and implicit as seen in military 
decision making, the M&S community needs to avoid the rhetoric that labels subjective 
modeling as non-optimal. Unfortunately, even after Ilachinski (2004), a well-known 
figure in the M&S community, noted that, “a major ingredient of modeling and 
simulation consists more of art than science” (30), most practitioners of M&S still tend to 
avoid the art of modeling for fear of inducing subjectivity into the study. And while the 
source of this fear is rooted in modeling VV&A, VV&A in itself is no excuse for 
producing inaccurate models. Models that are based on inaccurate referents will likely 
pass VV&A assessments, but they are still inaccurate, and achieving the VV&A 
certification should not be the primary goal of any MDP, an accurate model should be. 
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The development of models focused on achieving VV&A certification is more of a 
programmatic risk mitigation technique designed to provide developer’s protection and 
user’s assurances than an M&S MDP. And while VV&A is valuable for specific cases, it 
has produced a community of model developers that have become more focused on 
achieving VV&A certification than producing the best possible models for the intent of 
their studies. Thus, the community finds itself in an environment which often neglects 
potentially insightful model development techniques like subjective and implicit 
modeling that could potentially improve the accuracy of models. By focusing on 
codifying a methodology for executing subjective modeling in a disciplined and scientific 
manner, the M&S community as a whole would be better equipped to handle a larger 
range of potential modeling requirements. 

5. Implicit Modeling 

Another creative way to attempt to capture the impacts of ESINQ effects in 
models is through implicit modeling. While not a new concept, because of the recent 
increase in acceptability of implicit modeling practices and its potential for addressing 
ESINQ factors and effects within models, it will be consider an emerging modeling effort 
in this work. Simply defined, implicit modeling is the representation of an unknown, 
qualitative, or ignored function that has been included or aggregated as part of a 
quantitative element in a model. Most model developers execute implicit modeling as a 
byproduct of more traditional MDPs, where the “details that are included are said to be 
explicitly represented and the excluded detail is implicitly represented” (Cares 2004, 2). 
Unfortunately, there is little thought to how these implicitly modeled effects are chosen, 
and even less verification of their significance to the outcomes of the model. 

Unlike external effects, which can only represent known systems and quantifiable 
effects, implicit modeling could allow for the inclusion of all ESINQ factors and effects 
in models, whether known or unknown, quantifiable or not. In the past, implicit modeling 
has been heavily scrutinized because it fails to meet the first principles of modeling, 
specifically when considering the cause and effect so critical for model validation. But 
even under such scrutiny, the need for flexible modeling methodologies capable of 
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representing ESINQ factors has not gone unnoticed. When considering ESINQ effects 
and other soft factors, “first principles models are simply not adequate. Eor such 
phenomena, accurate prediction of outcomes, other than as aggregate probabilities, is 
simply a bridge too far” (Middleton 2010, 131). Thus, it may be time to start exploring 
the potential of models that do not uphold the first principles, and look at models that are 
more accommodating of the harder to quantify factors and effects like ESINQ effects. 
This need is best captured by a statement from the Navy: “there has been a chronic 
tendency for DOD modelers and analysts to avoid representing or considering ‘soft 
factors’ despite the fact that history tells us they are often dominant” (Committee on 
Technology for Euture Naval Eorces 1997, 26). Eor modelers who are interested in soft 
factors, as the author is with ESINQ effects, the subjective nature of assessing soft factors 
has forced the community to re-look at the potential utility of implicit modeling. It is 
obvious that first principle models are inadequate to address ESINQ effects, but does a 
model need to uphold the first principles for the insight generated for it to be useful? The 
author argues no, and that implicit modeling should not be dismissed outright. 

In terms of this research, there are three key aspects that support the use of 
implicit modeling. Eirst, is the fact that the ESINQ factors and effects of the most interest 
to this work are extremely difficult to represent in first principle models. The reason for 
this is the fact that the factors “we can measure easily do not capture critical intangibles: 
morale, leadership, unit cohesiveness and the like. Eurther complicating matters is the 
fact that generally, for a host of complex reasons, the whole is not just a simple sum of 
the parts” (Middleton and Mastroianni 2008, 4.2). Thus, the critical intangibles (ESINQ) 
of interest to this work do not lend themselves to traditional MDPs. Additionally, as 
noted by Middleton and Mastroianni (2008), traditional MDPs fail to account for the 
synergistic impacts that many intangibles have on the potential outcome of the model. 
This leads to the second reason implicit modeling shows potential, and that is the fact that 
implicit modeling has more flexibility to address the dependence of systems on ESINQ 
factors and effects for the generation of combat power. In traditional MDPs, this 
dependency would either be aggregated with the system in the model or ignored 
altogether, but with implicit modeling it should be possible to delineate the two, which 
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opens a host of opportunities with regard to model development, analysis, and 
applications. The third aspect that supports the use of implicit modeling is the fact that 
most ESINQ factors can be articulated in three ways. They can be modeled directly, or 
have their contributions to combat power linked to either SA or system attributes. 
Traditional MDPs can only model the known aspects of ESINQ factors directly in the 
model, ignoring the other two ways. An implicit MDP on the other hand can do all three: 
either directly as in traditional MDPs; by capturing the portion of the ESINQ factors that 
can be represented as system attributes, implicitly modeled as part of the system itself; or 
by capturing the SA piece implicitly through the use of surrogate factors in the model. 

While implicit modeling techniques are often overlooked because of their 
questionable subjectivity, it is the author’s belief that with a logical and well thought out 
methodology, an implicit modeling technique has more than enough advantages for 
consideration as a legitimate MDP. And as noted by Middleton (2014), such a model 
could “still fall under the purview of scientific rigor, but there is a need to extend that 
concept to incorporate a ‘soft,’ incremental focus, where parametric analysis bounds 
regions of factor effects and the extent/significance of functional relationships, and where 
increasing levels of correlation correspond to increased acceptance of predictive validity” 
(7). By applying effort here, in the formalized definition of an implicit MDP, it should 
provide M&S users another tool set for addressing problems that do not fit well into 
traditional MDPs. As discussed by Middleton and Mastroianni (2008), for modern model 
developers to break out of the “to hard to do” traditional MDP paradigm, “closed systems 
modeling approaches need to be augmented with a new more flexible modeling 
paradigm” (5.5), one capable of accepting ESINQ factors and effects and supporting the 
development of models that can provide more robust insight into the OE. 

6. DOE: Design Space versus Solution Space 

Another potential gap with regard to modem modeling practices has to do with 
the potential misuse of DOE. While DOE provides significant utility to analysts, offering 
a detailed and efficient means for conducting model analysis, there are potential issues 
regarding the interaction between design space and solution space that must be 


85 



considered. While most DOE analysis practices achieve design saturation during DOE 
development, they can sometimes fail to ensure that the solution space is saturated as 
well. This failure can result in an analyst assuming that the DOE output, as well as the 
meta-models generated through that analysis, will accurately represent the system of 
interest. Unfortunately, this is not always the case and depending on the level of solution 
space saturation, can sometimes result in meta-models that are skewed. To highlight this 
potential issue, eonsider the DOE scatter plot in Eigure 31. 



Eigure 31. Design Space for a Seven Eactor Order NOEH 

The design space of this seven factor order Nearly Orthogonal Eatin 
Hypercube (NOEH) design is highly saturated, ensuring that the interior space is being 
fully explored and is capable of identifying key interactions and dependencies of the 
factors. The 2"‘* order NOEH was developed by MacCalman et al. (2017) and as he states 
in his work, has three key advantages which are key to this work: “Eirst, they can fit the 
most commonly used polynomial metamodel with guaranteed minimal correlations; 
second, with suitable caution, they can fit higher-order models to a handful of factors; 
and third, they are space-filling allowing us to take full advantage of partition trees to 
find interesting behavior in local areas of the experimental region” (148). Thus, the 2"‘^ 
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order NOLH design allows for a more complete and thorough investigation of the 
solution space, while increasing the likelihood of identifying key system interactions. 
Following the execution of this design, analysis is conducted to draw out relevant insight. 
For this research, this process focused on establishing a baseline set of factor settings that 
would ensure that the model outcomes were calibrated to an expected victory curve 
across a range of potential Force Ratios (FRs). The analysis of the output from the design 
described in Figure 31 can be seen in Figure 32. 
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Figure 32. IMP Modeling of DOE Output Data 


Based on the analysis conducted, the attribute settings shown at the top of Figure 
32 should produce model outcomes that are within +/- 5% of the expected victory curve, 
across all FRs from 0.9 to 1.75. To verify, a one factor verification DOE was executed on 
the model with these agent attributes, where only the ER was varied. The plot of this 
verification can be seen in Eigure 33. 
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Figure 33. One Factor Verification DOE (Un-saturated) 


As shown in Figure 33, the verification run produced a mean outcome curve 
(Blue) that was vastly different than the expected victory curve (Red) based on the 
analysis of the DOE output. Based on the analysis and agent attributes described in 
Eigure 32, the model outcome curve should have been very similar (+/- 5%) to the 
expected victory curve, but this is not the case. In some instances, we see a variation from 
the mean of greater than 30%. This unexpected result highlights the issue with failing to 
ensure that the solution space is saturated. If a user of DOE fails to address the potential 
issues with regard to solution space saturation, it is possible for them to achieve skewed 
and inaccurate results as described in Eigure 36, and they may not even know it. Eigure 
34 describes the output from the original DOE, which was used to produce the meta¬ 
models used by the IMP contour profiler shown in Eigure 32. 
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Figure 34. Un-Saturated Solution Space for a Seven Factor Order NOLH 

Evidently, the design saturation shown in Figure 31 does not always ensure 
solution space saturation. In this example, only 12% of all Design Points (DPs) resulted 
in an outcome that fell within the range of interest. Thus, the analysis and meta-model 
development based on this un-saturated solution space produced skewed results that do 
not accurately represent the model, as we saw in Figure 33. An un-saturated solution 
space can be characterized by two general definitions. First, an un-saturated solution 
space will have less than 50% of the outcomes fall within the range of interest. Second, 
an un-saturated solution space will have an un-equal distribution of outcomes around the 
expected outcome curve, typically greater than a 10% deviation. Not only are the 
outcomes described in Figure 34 weighting heavily to the upper extreme, with 69% of the 
outcomes, but only 12% of the outcomes fall within the range of interest. Thus, using the 
author’s definition of saturated, the outcome solution space is highly un-saturated, and 
the meta-models developed from the analysis of this data will likely result in skewed 
results that heavily favor the upper extreme as FR increases, which explains the shape of 
the model output described in Figure 33. 

To avoid un-saturated solution spaces, users of DOE must take an iterative 

approach to DOE that focuses on reducing the dimensionality of the design. Through 

manipulation of the DOE ranges of each factor, the screening of factors for significance, 
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as well as increasing the design density through stacking, it is possible to increase the 
saturation of the solution space within the area of interest. A saturated solution space will 
support more accurate analysis, and once maximized, can produce much more accurate 
meta-models than achieved previously. Following DOE iteration and manipulation. 
Figure 35 describes a more saturated solution space. 
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Figure 35. Saturated Solution Space for a Seven Factor Order NOLH 


As depicted, the outcomes above and below the expected victory curve are 
relatively equally distributed, within 10% of the optimal 50/50 distribution. Additionally, 
at least 50% of the outcomes fall within the range of interest. Thus, the output described 
here is by definition saturated, and the analysis from this output should now produce 
meta-models that accurately represent the system of interest. To confirm, a one factor 
verification run was conducted, and the results of that analysis can be seen in Figure 36. 
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Figure 36. One Faetor Verifieation DOE (Saturated) 

Following the reeognition of potential issues regarding solution spaee saturation, 
and the implementation of the reeommendations laid out in this seetion, a more aeeurate 
meta-model ean be aehieved through an iterative proeess of DOE refinement that foeuses 
on redueing design spaee dimensionality and inereasing solution spaee saturation. Note 
that the model was not ehanged in any way, only the method in whieh the DOE was 
implemented. Thus, through a direeted manipulation of the DOE, a meta-model was 
developed that was capable of better modeling the system it aims to represent. 

D. SUMMARY 

This chapter provided the reader with the foundational knowledge needed to 
understand the problem the U.S. military faces from operations in a D3SOE. It started by 
describing U.S. space dependencies, major adversaries and their counter-space 
capabilities, the new OE, and the associated vulnerabilities that the United States faces 
with respect to emerging threats. Next, it added a significant amount of detail regarding 
current and emerging mitigation strategies. The purpose of this was to provide the reader 
linkages between current and emerging threats and U.S. mitigation strategies, and to set 
the stage for identifying the potential gaps, specifically with regard to modeling. Einally, 
it addressed the modeling gaps which the author believes to be directly responsible for 
the inability of the United States to effectively prepare for operations in a D3SOE. This 
chapter highlighted the inability of current MDPs to capture the actual OE, as well as 
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addressed the underlying modeling issues, specifically discussing 1*^' and 2"‘^ order gaps, 
domain segregation, error versus complexity, subjectivity, and the inability of current 
MDPs to quantify the impacts from ESINQ effects. It is this author’s belief that through 
the development of an improved MDP that can address ESINQ effects, it will be possible 
to capture a more accurate representation of the OE in a model, and break the trend of 
developing l'^* order systems, as shown in Eigure 37. 



Eigure 37. Methodology for the Development of 2"‘^ Order Systems 

By developing a methodology that can more accurately address ESINQ effects 
within a model, it will be possible to capture a more holistic understanding of the OE, and 
after sharing that understanding between the four referents, it should be possible to close 
the and order gaps of D3SOE mitigation. By making operational and acquisitions 
decisions based on the performance of competing emerging space systems and strategies 
within this more complete understanding of the OE, we greatly increase the chances for 
success by producing a more robust system with direct traceability to metrics of 
operational effectiveness. Unfortunate, although the necessity for a better MDP has been 
recognized, the methodology needed to “build the bridge” from where we are 
(unprepared) to where we want to be (prepared) has not been codified. 
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III. AN IMPLICIT MODEL DEVELOPMENT PROCESS 


The moral is to the physical as three to one. 

—Napoleon (quoted in Moore 2017) 

This research expands traditional MDPs by developing an IMDP that can more 
accurately address the modeling gaps described in Chapter IT The overarching goal of 
this work is to improve the decision-making process of leaders in both the operational 
and acquisitions communities by providing a more robust analysis methodology that can 
support the development of more accurate decision support tools. As stated in the 2009 
National Intelligence Strategy, “being able to deliver capability cost-effectively when it is 
needed improves mission effectiveness, provides leadership with flexibility in making 
investments, and precludes gaps in necessary capabilities” (The Office of the Director of 
National Intelligence 2009, 16). The IMDP described in this chapter will provide analysts 
the means to simultaneously evaluate an emerging system’s performance across all four 
of the potential contributions to a models referent, and allow us to more thoroughly 
explore the four referents of D3SOE mitigation described in Chapter II. Because ESINQ 
factors are typically considered external to the system boundary and non-quantifiable, 
they have been largely ignored in the past. Yet ESINQ factors are very similar to 
Napoleons view of moral in the quote at the beginning of this chapter, and while they 
may be difficult to measure, can often have a significant impact on the outcomes of the 
model. The IMDP presented in this work will give users the ability to loosely quantify or 
“bound” the impacts from ESINQ factors on measures of operational effectiveness. The 
IMDP will facilitate a more complete understanding of a system’s performance with 
respect to the OE and thus, support the development of a range of new and improved 
operational and acquisitions decision support tools. These improved tools can be used to 
provide decision makers with a better representation of the OE by more accurately 
accounting for the impacts of not only the system, but of the ESINQ effects on the system 
as well. By following the IMDP presented here, a better representation of the OE can be 
achieved, which will allow for more informed operational and acquisition decisions 
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regarding the allocation of resources. Before we discuss the IMDP further, a brief review 
of MBSE and trade-space exploration methodologies is in order. 

A. MBSE 

MBSE is a relatively new concept within the SE community best covered in the 
works of Wymore (1993), Eriedenthal, Moore, Steiner (2013, 2015), and Eaw (2014). 
The International Council on Systems Engineering (INCOSE) defines MBSE as the 
“formalized application of modeling to support system requirements, design, analysis, 
verification, and validation activities beginning in the conceptual design phase and 
continuing throughout development and later life cycle phases” (International Council on 
Systems Engineering 2007, 15). MBSE differs from traditional SE in that it takes a more 
upfront and formal approach to the use of modeling in the SE process. In fact, the only 
difference between the INCOSE MBSE and SE definitions is that MBSE highlights the 
“formali z ed application of modeling” in support of the SE process. Yet the significance 
of this slight variation is profound. MBSE attempts to replace many of the older 
“documents based” processes common within the SE communities with a more “model¬ 
centric” approach to SE. As technology has advanced, so has the accuracy and 
complexity of most systems architecture software packages, which have begun to surpass 
the capacity of most document based SE processes to maintain an accurate and complete 
record of the system. To address this shortfall, models are taking a more essential role in 
modem day SE practices, replacing the analog tools of the past that can no longer keep 
pace. Eigure 38 shows a simple graphical representation of the linkages between MBSE, 
TSE, and M&S. 
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MBSE increases efficiencies 
within SE processes by putting a 
more formal and upfront 
emphasis on the use of M&S in 
systems design 



Figure 38. SE Analysis Methodology 


From the author’s perspective, MBSE uses M&S and TSE techniques to better 
inform the SE process, attempting to expand the overlap of M&S and TSE within the SE 
community. The overarching goal of MBSE is to improve the efficiency of the SE 
process through better integration of modeling earlier and throughout the systems life 
cycle. The major benefits of employing MBSE as opposed to traditional SE are 
highlighted in the work of (Friedenthal, Moore, and Steiner 2015, 20), which describes 
the six primary benefits of MBSE. These include enhanced communications, reduced 
development risk, improved quality, increased productivity, the leveraging the models 
across the system life cycle, and enhanced knowledge transfer. Thus, it is easy to see the 
advantages that MBSE can bring to system development programs. A major theme in 
modem acquisitions, especially in the DOD, is the need to develop more efficient 
processes that reduce risk and cost while delivering more capable systems through the 
use of “streamlined processes to improve readiness and speed acquisition” (U.S. Army 
2014b, 20). Of the six primary benefits of MBSE, three deserve additional consideration. 

The first benefit of MBSE to acquisition programs is reduced developmental risk. 
By framing a more complete understanding of the system with respect to its OE, program 
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managers are able to better estimate system programmatics during the conceptual design 
phase and thus, reduce the risk of cost and schedule overruns, which are all too common 
in modern acquisition programs. The second benefit is improved quality. By 
implementing the MBSE process, organizations should be able to produce more capable 
and resilient systems that better meet the requirements of the stakeholders across a wider 
range of OEs. Because the MBSE process emphasizes a more upfront investment of 
M&S than traditional SE processes, especially during the concept development phase, 
MBSE provides greater traceability of system requirements across all phases of the SE 
process, ensuring a more complete and unambiguous understanding of the system. The 
third benefit of MBSE is increased productivity. Because MBSE supports the building of 
a more accurate understanding of the system and its requirements earlier in the system 
life cycle, the need for iterative design is significantly reduced, yielding a more timely SE 
process. Additionally, by more fully exploring the system trade-space and conducting 
system effectiveness analysis earlier in the system life cycle, the chances of producing a 
more capable system are increased. Thus, the development of a better understanding of 
the interactions of the system with the OE during the conceptual design phase as well as 
the potential trade-offs among the system alternatives is critical. And as noted by 
MacCalman et al. (2015), by focusing on a data driven approach to design like MBSE, 
“the end result will be better informed decisions, faster engineering, less rework, and 
allow for a wider range of alternative solutions” (5). 

In addition to the benefits that MBSE brings modern acquisitions programs, there 
is a relatively new concept in current MBSE analysis methodologies that is of 
considerable interest to this work, and that is the concept of OEM. Traditionally, the SE 
process attempts to achieve stakeholder needs by developing an appropriate design 
through system tradeoffs of MOPs and iteration, and then by measuring that design 
against MOEs in the appropriate OEs. Unfortunately, even after creating numerous 
system designs during alternative generation, there is no way to ensure that the most 
robust designs will be captured. Eoosely defined, a robust design refers to a solution that 
provides a more stable outcome across a wider range of potential environments. An 
optimal design on the other hand provides the best possible outcome under a very specific 
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set of circumstances. Unfortunately, this specific set of circumstances typically only 
accounts for a small portion of the actual OE in which the system will likely perform. 
Thus, robust designs are often more desirable in modern SE practices where a great deal 
of uncertainty resides, and while they typically do not perform as well as an optimal 
system, they do so more stably across a much larger range of potential environments than 
optimal systems. The inability of traditional SE methodologies to capture the most robust 
designs is due to two major reasons. Eirst, MOPs rather than MOEs tend to drive system 
design choices, which are highly biased due to the dependency on human expertise. 
Second, traditional SE approaches cannot generate a sufficient number of design 
alternatives to ensure the solution space is saturated. In his work. Brown (2013) 
highlights that “an early structured search of the design space through the synthesis and 
assessment of hundreds or thousands of alternative concepts is essential for sufficient 
understanding of the relationship among cost, effectiveness, and risk” (10). To address 
this shortfall, some emerging TSE techniques and MBSE analysis methodologies have 
surfaced that use OEM to take the reverse approach, using MOEs to drive design 
decisions by establishing traceability between a systems design characteristics and its 
operational effectiveness. Thus, a more thorough exploration of potential design 
alternatives can be made, which is much more likely to capture a more robust system 
design and produce a more capable system. 

B. TRADE-SPACE EXPLORATION 

The purpose of this section is to provide the reader with a brief introduction to 
some of the recent advancements in SE analysis methodologies that have shown potential 
for improving the capabilities of SE and MBSE process to support better TSE. 
Specifically, this section discusses the work of Dr. Alex MacCalman (2013) and his 
efforts in developing a state-of-the-art 2"‘^ order DOE tool that allows for a more 
complete exploration of a system’s trade-space by more accurately accounting for the 
higher-order interactions of a system. While most SE processes execute TSE at some 
level to support better decision making, these process are often constrained by 
complexity, limiting the breadth and depth of the analysis of the solution space to just a 

few alternative designs. While the work conducted by Dr. MacCalman focused on the 
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creation of a genetic algorithm, followed by the creation of the DOE design tool that used 
this algorithm to produce new designs, the aspect of his work that most interested the 
author was how these improved designs could be used to enable a more thorough 
exploration of the trade-space. MacCalman et al. (2016) highlight how advanced DOE 
can be used to execute TSE more effectively, enabling engineers to “simultaneously 
explore the operational and physical domains using statistical surrogate models in order 
to illuminate trade decisions between the system’s operational effectiveness and physical 
design considerations” (1). By synchronizing both operational and synthesis models 
through the use of meta-models that captured a more accurate understanding of the OE, it 
was possible to more accurately link a system’s functions to physical components. This 
improved understanding allowed for a more complete visualization of the trade-space and 
allowed users to observe the impacts to system performance based on changes to the 
functional and physical architectures. It is this improvement that is at the heart of the 
author’s interest in his work. His efforts expanded the use of TSE to more accurately link 
M&S and MBSE, and as seen in Eigure 39, enabled a more accurate representation of the 
OE during system design. 



Improving MBSE through the 
developmentand application of 
DOE for use in trade-space 
exploration during system 
conceptual design 


Eigure 39. MacCalman’s Expansion of SE Analysis Methodologies 
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As shown in Figure 39, the use of advanced M&S techniques, including DOE, can 
support a more complete and accurate TSE during the MBSE process. By improving the 
quality and density of the data feeding MBSE analysis, a more accurate representation of 
the system’s interactions with the OE can be characterized, highlighted by an increased 
overlap and yielding a more insightful TSE than previously possible. Yet, even with the 
ability to more accurately capture the OE through the use of advanced DOE, MBSE 
analysis methodologies still lacked a formalized process to implement TSE in 
conjunction with system architecture steps and products, specifically with the integration 
of these synchronized products into external models. 

C. MBSE ANALYSIS METHODOLOGIES 

The purpose of this section is to provide the reader with a brief introduction to 
MBSE Analysis Methodologies, specifically the MBSE MEASA developed by Beery, 
which expanded upon the work of MacCalman. In his Dissertation, Beery describes 
various shortcomings of the then current SE and MBSE analysis methodologies which 
could limit a system engineer’s ability to fully describe a system. These gaps have been 
binned into four overarching shortcomings: unsynchronized SE architectures, limited 
linkages between Systems Modeling Eanguage (SysME) products and M&S, the inability 
of current analysis methodologies to support TSE, and the lack of SE architecture 
linkages to operational effectiveness. The intent of the MBSE MEASA was to address 
these shortcomings and support a better understanding of the system through the 
“analysis of models and simulations that consider not only system design attributes (as is 
done in each of the MBSE methodologies presented in the previous chapter) but also 
environmental and operational factors during system conceptual design” (Beery 2016, 
66). The MBSE MEASA formally defines the process for ensuring synchronization 
between the functional and physical architecture processes and the development of 
SysME products for integration into the external models which will be used to access 
system performance, all while maintaining traceability to stakeholder requirements. By 
doing so, the MBSE MEASA successfully “establishes the formal linkage between 
operational need and physical system configuration that should be the focus of any 

MBSE based analysis methodology” (Beery 2016, 71). Eike all MBSE analysis 
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methodologies, the MBSE MEASA captures all the key SE process necessary to fully 
describe a system, but more importantly, the MBSE MEASA provides a means in which 
to translate those physical and functional descriptions to the external model, which had 
not previously been codified. Beery was able to demonstrate the utility of his contribution 
by applying a set of SysME products of an emerging mine warfare system to the 
development of an improved mine warfare model. Then, through the use of advanced 
DOE and TSE techniques demonstrated by MacCalman, use the model to generate 
detailed data to support robust analysis needed to investigate the effectiveness of the 
emerging system compared to the legacy systems. Eor more detail regarding the MBSE 
MEASA please refer to (Beery 2016). Eigure 40 shows a simple diagram of how Beery’s 
work expanded the capacity of MBSE to support SE and TSE. 



Improving MBSE and trade-space 
exploration through the 
formalized application and 
linkage of SysML architecture 
\ products with external M&S 


Eigure 40. Beery’s Expansion of SE Analysis Methodologies 


Beery’s work allows for an expanded integration between MBSE and M&S to 
support better TSE, denoted by the increased area of overlap. By formalizing the process 
in which architecture products can be synchronized for integration into external models, a 
more accurate representation of the systems interactions with the OE can be characterized 
in MDPs. Thus, as before, the overall understanding of the system and its interactions are 
improved, yielding more insightful TSE than previously possible. It is because of this 
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improved capacity to capture a more accurate understanding of the system, the translation 
of that understanding to operational and synthesis models, as well as its robustness for 
exploration of the system trade-space through OEM, that it was selected for use as the 
basis for the acquisitions support tool which will be described in Chapter V. With that 
said, there remains some significant shortcomings in current TSE and MBSE analysis 
methodologies, specifically with regard to MDPs, that limits the usability and 
effectiveness of current methodologies to consider the impacts of ESINQ factors. 

D. AN IMPLICIT MDP 

This research addresses what the author believes to be a few critical shortcomings 
of current M&S practices and techniques that have significantly limited the effectiveness 
of M&S to support modern TSE. The goal of the IMDP is to address these shortcomings 
by capturing a more accurate representation of the OE (the portion of the real world that 
comprises the context for the model) and its impacts on system performance. By doing 
so, the IMDP should be capable of better accounting for the model development gaps 
identified in Chapter II and result in a more complete understanding of a system and its 
interactions with the OE during the SE process. However, before the IMDP is defined in 
detail, a brief re-cap of the shortcomings of current MDPs and MBSE analysis 
methodologies is in order. 

I. Background 

While modem MBSE analysis methodologies provide a robust methodology for 
building a better understanding of the system and its interactions through the 
synchronization of systems architecture products with M&S, it fails to address the effects 
that ESINQ factors have on system metrics of operational effectiveness. This failure 
results in an exploration of the system trade-space based on an incomplete understanding 
of the OE. Thus, by developing a methodology that supports the inclusion of ESINQ 
factors within the model, current MDPs will be improved. During the investigation of 
traditional MDPs in this work, two primary gaps and one secondary gap were identified 
that the author believes could be addressed through the expansion of the current MDPs 
and MBSE analysis methodologies to capture a more accurate understanding of the 
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system and its interactions with the OE. The primary gaps were based on the authors own 
observations of the state-of-the-art, and focused primarily on the bounding of ESINQ 
effects within the MDP, as well as the creation of a formalized methodology for 
identification and selection of appropriate M&S packages, the MSSSM. Because the 
MSSSM is more generalized than the implicit modeling of ESINQ factors that is of 
primary focus, it is outside of the scope of this work, and its development will not be 
discussed in any detail here. Eor more detail regarding the MSSSM, please refer to 
Chapter I and Appendix A. The secondary gap was based on the future work section of 
Dr. Beery’s dissertation as well as the author’s desire to support the acquisitions of 
emerging space systems. Here, the focus was on the expansion of current MDPs and 
MBSE analysis methodologies to address the modeling of non-traditional systems, which 
ties directly to a contribution of this work, which was to use M&S to explore the capacity 
of emerging systems to mitigate the operational impacts from operations in a D3SOE. Eet 
us briefly discuss both of these gaps, starting with the bounding of ESINQ effects. 

While current MDPs and MBSE analysis methodologies support the development 
of more accurate system definitions, neither can adequately address ESINQ factors and 
their impacts on the system. In an age when network-centric operations of highly 
technical systems has become the standard, little has been done to fully understand or 
capture the dependencies that modern systems have on ESINQ factors to generate 
internal metrics of system effectiveness. Simply put, as modern systems continue to 
evolve increased dependencies on external elements, our understanding of those systems 
will continue to diverge from the ground truth because we cannot accurately attribute or 
quantify the impacts of those external elements. The issue with this divergence is similar 
to the issues revolving around the inability of most combat models to account for soft 
factors like moral and leadership. This failure is best captured by a statement from the 
Navy which stated that “this disjunction between model and reality has long undercut the 
credibility of most combat models with warriors, historians, and analysts willing to 
recognize soft factors and uncertainty” (Committee on Technology for Euture Naval 
Eorces 1997, 26). The inability to account for soft factors, which are included in the 
definition of ESINQ, has created an environment where modem models routinely ignore 
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potentially significant contributions to a systems’ total operational effectiveness. This 
produces models that can often underestimate the operational effectiveness of the systems 
being modeled, and can be linked to two primary causes. First, when external sources that 
contribute to a systems internal operational effectiveness (MOPs) are ignored, leaving 
part of its total contribution unaccounted for in the overall system assessment. Second, 
when the external sources that contribute to the systems overall external operational 
effectiveness (MOEs) are ignored. Thus, the models developed using modem MDPs are 
inherently inaccurate. And though George Box famously said as far back as 1976 that 
“All models are wrong,” the point here is that current MDPs produce models that are 
“more wrong” than they could/should be. By ignoring some potentially significant 
modeling input sources, traditional MDPs fail to capture a more accurate representation 
of the OE. As noted by the Committee on Technology for Euture Naval Eorces (1997), 
this divergence from reality has undermined the credibility of most models with users. 
These flaws derive from the failure of the model to capture the contributions of ESINQ 
factors on measures of system effectiveness, as measured both inside (MOPs) and outside 
(MOEs) of the system boundary. Unfortunately, as the dependencies of modem systems 
on external support continue to rise, so will the inaccuracies of traditional models. Now, 
let us discuss the secondary gap that the author believes could be addressed through the 
expansion of the current MBSE analysis methodologies and MDPs, specifically, the 
modeling of non-traditional systems. 

Potential systems current under investigation for mitigating operational risks from 
adversary use of counter-space capabilities lies in the development of emerging space 
capabilities like SmallSats and HAAS. These systems are conceptually complex, being 
designed to operate based on multiple operational concepts, with sometimes contradicting 
missions. Thus, the assessment of these systems and their capability to mitigate 
operational risk is difficult to quantify because not only are there sometimes differing 
requirement, but most models are unable to account for the majority of the external 
dependencies of a system as well. Prom one perspective, the system is assessed from a 
traditional SE approach; where the system’s capacity to execute system level functions is 
weighed again internal system level MOPs and external MOEs. Yet on another hand, the 
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secondary roles of the system is assessed from a non-traditional approach, where the 
system’s capacity to effect the overarching organizational structure, the System of 
Systems (SoS), is the primary metric for measuring internal SoS MOPs and external SoS 
MOEs. Thus, the increased complexity of the external interactions and dependencies of 
these systems as well as the variable means for which the system is assessed leads to their 
classification as non-traditional systems. Non-traditional systems can be loosely defined 
as systems that cannot be easily modeled or assessed using common and widely accepted 
M&S and SE processes. As stated by Beery (2016), “the most direct contribution consists 
of expansions and redefinitions of the MBSE MEASA to non-traditional systems 
(systems with limited control over design as well as systems that exhibit emergent 
behavior are potential examples, although others may exist)” (186). Non-traditional 
systems require a different approach to design than traditional systems because their 
capabilities are often not measured at the system level; rather, they are often measured at 
both the system and SoS levels. Simply stated, the primary MOEs of these non-traditional 
systems define how well they perform their mission in the context of the system, as well 
as how well they support the MOEs of the larger SoS. Thus, the assessment of the 
systems effectiveness is difficult to model because of the differing and sometimes 
competing requirements between the system and SoS. An expanded MDP and MBSE 
analysis methodology that can account for the dependencies of a system on ESINQ 
factors could help define a better understanding of the linkages and dependencies of non- 
traditional systems on the external environment and the SoS. 

Now that the major shortcomings of current MDPs and MBSE analysis 
methodologies have been discussed, it is appropriate to discuss where these shortcomings 
can be addressed. To highlight these shortcomings, consider the generalized MDPs 
described in Chapter II. Simply speaking, the majority of the gaps and shortcomings of 
modem models reside in the MDP, specifically, the model definition step. As shown in 
Eigure 41, the IMDP addresses these shortcomings in three major ways. 
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Figure 41. Applying the IMDP to the MDP 


First, part A of the IMDP formalizes the process for bounding ESINQ factors by 
better defining the interactions of the system with the external environment during model 
definition. Part A expands and provides more structure to the weakly defined model 
definition step of traditional MDPs, and provides a means for accounting for both 
external dependencies and ESINQ effects in the model referent. Second, part B of the 
IMDP formalizing the process for implicitly representing this improved definition during 
model development, and supports a more accurate representation of the OE in the model, 
part B defines the implicit modeling process for iteratively implementing the improved 
referent developed in part A using surrogate factors, and enables the calibration and 
modeling of external dependencies and ESINQ effects. Third, and while a standalone 
process from the IMDP, a formali z ed methodology for screening and selecting of an 
appropriate M&S packages is provided, which guides the user to the selection of a more 
appropriate M&S packages, a step that is typically ignored during MDPs. The 
combination of these three improvements to current MDPs allows for a more complete 
understanding of the system, to include external dependencies, ESINQ effects, and non- 
traditional systems and their interactions with the environment within the model, as well 
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as a more complete investigation of the system trade-space. Now let us look at how the 
traditional MDPs were expanded to create the IMDP. 


2. Overview 


The end state of this research is to improve current MDPs and MBSE analysis 
methodologies by formalizing the process for accounting for ESINQ factors within a 
model. By accounting for ESINQ factors during model development, the resulting 
models will more accurately describe the system and its interactions with the OE, and 
provide the user more novel insights into the operations of the system of interest. Thus, 
through implementation of the IMDP, it is possible to bound and capture ESINQ factors 
which were largely ignored in the past, yielding a more accurate representation of the OE 
within the model. This more complete understanding of the system could then be 
incorporated into MBSE analysis methodologies by integration and use of these more 
refined models, and thus, provide a more accurate data set from which analysis can be 
performed. This is demonstrated in Eigure 42, which highlights the improved 
synchronization of M&S, MBSE, and TSE through the execution of the IMDP. 



Improving MBSE and trade-space 
exploration through the 
formalized inclusion of ESINQ 
factors into M&S to improve 
decision support tools 


Eigure 42. Pugsley’s Expansion of SE Analysis Methodologies 
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The IMDP developed in this work further expands the efforts of MacCalman and 
Beery by formalizing a MDP that can account for ESINQ factors, and then using this 
improved model to support a more accurate MBSE process. This improved understanding 
of the OE will result in more accurate model development, and thus, a more accurate and 
complete MBSE and TSE processes. The key to achieving this outcome, as well as the 
other secondary contributions discussed in Chapter I, is the ESINQ-enabled model. 
Unfortunately, as depicted in Eigure 22, most traditional MDPs do not account for 
ESINQ factors because they reside outside of the internal and external model input 
sources typically considered during model development. Thus, in order to achieve the 
ESINQ-enabled model that is central in obtaining the outcomes of this dissertation, an 
implicit MDP was needed to formalize the process for expanding the model input sources 
to capture/bound ESINQ factors within a model. The IMDP developed in this work 
serves to achieve this need, and by following the conceptualized process outlined in 
Eigure 43, model developers can produce ESINQ-enabled models. 


Implicit Model Development 
Process (IMDP) for Bounding 
External, Seemingly 
Intangible/Non-quantifiable 
(ESINQ) Factors 


Identification of ESINQ 
Effects and Translation to 
quantifiable internal impacts 1 


M&S Exploration for Suitability 



3 ., 


Normalization and Calibration 
of Mission Attributes to the 
selected assessment tool 
(IRCPAT) 


Calibration of Agent 
Attributes to the Improved 
Force Analysis System 
Weights Spreadsheet 


Application of Meta-models to 
improve operational decision 
support tools 


ESINQ Enabled Combat Model 


Manipulation of Improved 
model (with ESINQfactors) 
using quantifiable ESINQ 
impacts 


Base Model Realization 
(without ESINQfactors) 
and calibration to the 
IRCPAT using Agent 
Attributes 


A 



Eigure 43. IMDP Conceptual Plow 
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The IMDP conceptual flow depicts a generalized framework for capturing ESINQ 
factors of interest and translating them into the model. This conceptual flow consists of 
five general processes, and includes Translation, Normalization, Calibration, Realization, 
and Manipulation. Application, although noted in Figure 43, is outside the scope of the 
IMDP, however will be explored in Chapter V. This conceptual process is similar to 
Beude’s (2000) description of Forsberg and Mooz’s “Vee” diagram (10), in that the 
“decomposition phase” focuses on the process of breaking down, identifying, and 
quantifying ESINQ factors (functions that the model must represent), while the 
“integration phase” focuses on the building up, integration, and expansion of the model 
(synthesis representation of those ESINQ functions) to develop the ESINQ-enabled 
model. This conceptual flow diagram captures the essence of what the IMDP is 
attempting to accomplish, and through expansion and clarification of its terms, which can 
be seen in Table 3, will serve as the foundation for the contribution of this work. 
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Table 3. IMDP 


Part A 


Part B 


la 

Ust and describe the primarY exterrul, ar>d seemingly intangible and non^uantihable 
effeas in which you are trying to represent. (Use CEMM) 



lb 

dentify and bst the tangible impacts on combat operations which the effects in Step A 
would bkelv have. (Update EEMM) 

ic 

Using the MSSSM. investigate potential Combat Modeb and identify a Model with 
appropriate factors wtuch can best model the tangible impacts from Step 4B. that can 
be used as surrogates for the intangible effects. For the rest of this methodology I wil 
be using MANA as my M&$ package. (Update EEMM) 


2a 

Selection of a reference tool 



Base Lme the Force Analysis System We^hts Spread sheets to account for C4i 

2b 

contributions to combat power, then break this contribution into both external and 
internal contributions (if using the improved Relative Combat Power Assessment Tool 
(IRCPAT). skio this step, this has already been done. 





3a 

Baseline Cahbration. Build a simplified model and instantiate the reference tool. 


3b 

Calibrate the ‘Mission Attributes” to establish the baseline mission specific attributes 
for each agent for the selected model and mission set. 

3c 

Calibrate the ‘Agent Attributes” for each agent to account for variations m 
capabilities based on the improved Force Analysis System Weights Spread sheets to 
create the final ‘Agent Attnbutes” to be used m the Model. 




3d 

Calibrate the ‘ External Depersdency Attr^es” for each agent to account for internal 
dependencies on external sources for the generation of internal combat power to 
create the final ‘ ExtDep Attnbutes” to be used mi the Model. 


3e 

Calibrate the ‘ESINQ Attnbutes” to establish the baselme ESiNQ specific attnbutes for 
each agent for the model. 

la 

Oevelope the model to improve its capacity to meet user needs and requirements, 
without any modification to the ESINQ factor surrogates from 
Mission/Agent/ExtOep/ESiNQ calibration identified mi Part A. This is an iterative 
process where you build model capacity by adding mi other modeling aspects ignored 





during calbration, bke other forces. Conduct analysts as needed to adjust and fine tune 
the model. 1000 iterations of the model should be cahbrated to withm (♦/• 5%) of the 
output from the IRCPAT. To verify, execute a 1 x factor OOE to establish a analysts 
reference point for future comparison. 



2a 

Expand the model to Miclude the : ^ . factors identified mi Part A by modify the 

VC factors from their baseline values to a setting which achieves the expected 
impaas from each - .effea. This task supports the development of operational 
support tools. 


2b 

Execute a ful 2** Order NOIH DOE across all Force Ratios to establish a new analysts 
reference point, used to quantify the operational impacts from the l v . ef^ts of 
interest across all fuM rante of seventy. Same OOE process as descnbed in Part A3b-e. 

2c 

Quantifying ESI NQ Effects: Data analysis and Meta-model development for the 
responses of the i' NO factors, quantifyii^ operational impacts. Same Analysis 
process as descnbed m Part A3b-e. 


Translation 


Normalization 


Calibration 


Realization 


Manipulation 


The IMDP expands traditional MDPs by providing a more formalized 
methodology for model development by expanding the level of guidance and detail 
driving model development, specifically with regard to the model definition. For 
consistency with traditional MDPs the IMDP is broken down into two general steps. The 
model definition step (part A: decomposition) describes the process where ESINQ factors 
are identified, translated, normalized, and calibrated in preparation for inclusion into the 
model. This is where the shortcomings noted earlier are addressed, and through a 
formalized IMDP, will provide the ESINQ-enabled data that will drive the model 
development in part B. The model development step (part B: integration) describes how 
the model is developed and then expanded to integrate the ESINQ factors refined during 
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part A, to include the generation of meta-models to drive tool development. Each of the 
steps described here has an additional sub-sheet associated with it that clearly articulates 
the process for the user to execute the IMDP, which will be described in detail later in 
this work. The intent of the IMDP is to expand traditional MDPs by supporting a broader 
investigation of a given system and its OE, specifically enhanced to address and bound 
ESINQ effects within the model. This improvement greatly enhances the usability of 
current MDPs, and allows users to capture a more complete understanding of the system 
of interest, as well as its interactions with the OE and ESINQ factors. With a general 
overview of the IMDP completed, a more in depth description of the IMDP is needed to 
provide the required detail needed to fully execute the methodology. 

3. Presentation 

To avoid any unnecessary duplication between this description and other parts of 
this work, a more generalized approach to presenting the IMDP is taken here. 
Specifically, this chapter provides the reader a complete introduction to the steps of the 
IMDP in order to provide a solid understanding of the process, but will forgo any 
significant detail. Additionally, to keep this description concise, the products described 
here will provide only enough detail to articulate the conceptual processes, and are not 
meant to serve as executable documents for the development of the model. In later 
chapters. Chapter IV: IMDP Demonstration, and Chapter V: IMDP Application, the 
description provided here will be expanded to include more detail. 

a. Part A: Model Definition 

The purpose of the model definition step is to provide a formalized methodology 
that enables model developers to produce translatable definitions of ESINQ factors of 
interest for inclusion into models. Because of the nature of ESINQ factors as described in 
Chapter II, they cannot be directly implemented into models due to the inability to 
translate them into a form acceptable by most models. The majority of models simply 
lack the freedom in attributes to accurately model factors that were not envisioned during 
the models original development. Once you attempt to use a model for something other 
than the original purpose, like modeling ESINQ factors, you are limited to the original 
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attributes, which are more often than not inadequate for the unforeseen purpose. This is 
further complicated by the fact that the majority of ESINQ factors reside in domains 
outside of the ground domain (the context for which defines the referent for ground 
combat models), which induces a significant amount of aggregation and simplification 
errors due to cross domain translation. To clarify this, consider the inclusion of a satellite 
into a ground combat model. To model a satellite in a ground combat model would be 
difficult to say the least, primarily because even though the model may have 100s of 
attributes that can be used to accurately characterize a tank or artillery, these attributes 
are typically not very useful for defining a satellite. Not only does the combat model lack 
adequate factors and settings to accurately represent the satellite, because the effects of 
the satellite are typically measured using MOEs from a different domain, the impacts of 
that satellite on the combat model are difficult to quantify as well. When attempting to 
represent ESINQ effects in models directly, these cross domain translation issues 
typically yield an undesirable representation of the OE, which often preclude their 
inclusion. The general steps of part A of the IMDP are described in Eigure 44. 
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Eigure 44. IMDP (Part A) 

Part A of the IMDP expands the capabilities of current MDPs and MBSE analysis 
methodologies by formalizing the process for “bounding” ESINQ effects in a manner 
which more accurately translates to models than traditional MDPs allow, thus increasing 
the accuracy of the models referent without increasing the models complexity. The IMDP 
does this through the formali z ed translation, normalization, and calibration of modeling 
input factors with the goal of improving the overall accuracy of the model through a 
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directed manipulation of the supporting data. A detailed description of part A of the 
IMDP can be seen in Table 4. 


Table 4. IMDP Part A 


Part A 


la 

List and describe the primary external, and seemingly intangible and non-quantifiable 
effects in which you are trying to represent. (Use EEMM) 


lb 

Identify and list the tangible impacts on combat operations which the effects in Step A 
would likely have. (Update EEMM) 

ic 

Using the MSSSM, investigate potential Combat Models and identify a Model with 
appropriate factors which can best model the tangible impacts from Step 4B. that can 
be used as surrogates for the intangible effects. For the rest of this methodology 1 will 
be using MANA as my M&S package. (Update EEMM) 


2d 

Selection of a reference tool 


2b 

Base Line the Force Analysis System Weights Spread sheets to account for C4i 
contributions to combat power, then break this contribution into both external and 
Internal contributions (if using the Improved Relative Combat Power Assessment Tool 
(iRCPAT), skip this Step, this has already been done. 

3a 

Baseline Calibration. Build a simplified calibration model and instantiate the reference 
tool. 


3b 

Calibrate the "Mission Attributes" to establish the baseline mission specific attributes 
for each agent for the selected combat model and mission set. 

3c 

Calibrate the "Agent Attributes" for each agent to account for variations in 
capabilities based on the improved Force Analysis System Weights Spread sheets to 
create the final "Agent Attributes" to be used in the final Model. 

3d 

Calibrate the "External Dependency Attributes" for each agent to account for internal 
dependencies on external sources for the generation of internal combat power to 
create the final "ExtOep Attributes" to be used in the final Model. 

3e 

Calibrate the "ESiNQ Attributes" to establish the baseline EINQ specific attributes for 
each agent for the selected combat model. 


Translation 


Normalization 


Calibration 


Part A of the IMDP can be broken down into three primary processes, and include 
Translation, Normalization, and Calibration, which can in turn be broken down into 
lower level sub-steps (parts Ala - A3e). While these lower level sub-steps and their 
associated products will only be briefly described here, they will be demonstrated in 
detail in Chapter IV, and the link to the complete IMDP can be found in the 
Supplemental of this work. The intent of this section is to provide the reader with a 
description of the conceptual flow described in Figure 46, specifically by providing more 
detail regarding the functions of each of the three primary steps. For brevity, the complete 
demonstration of the process will not be documented here; any tables and figures 
discussed here are for explanatory purposes and are not meant to represent the fully 
articulated process of part A, which will be demonstrated in the next chapter. 

(1) Al: Translation 

The intent of this sub-step of the IMDP is to translate what would be considered 
an ESINQ factor or effect into a form which can be understood and implemented by the 
model. As described in Table 4, Translation has three primary steps, and begins with the 
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identification of the ESINQ effects in which the user is interested in accounting for 
within the model. Once these ESINQ effects have been identified, the IMDP leads the 
user through the mapping of these ESINQ effects to tangible effects and outcomes that 
can potentially serve as surrogates for the ESINQ effects within the model. Once these 
tangible effects have been identified, the IMDP directs the user through a formali z ed 
M&S package screening process where a suitable M&S package is selected that can best 
meet both the users’ primary intent for the study as well as address the ESINQ effects of 
interest. Once the M&S package is selected, the tangible effects which were mapped 
earlier are assessed for applicability to the selected M&S package, appropriate factor 
surrogates are selected. A brief overview of these three steps follows. 

Ala: Identifying ESINQ Effects and Expected Impacts 

The intent of this step is to build an understanding of the ESINQ effects in which 
the user is trying to implement in the model, as well as the expected impacts that these 
effects would likely have. This step has three primary sub-steps which facilitate an 
expanded understanding of the ESINQ factors of interest, and begins with the user 
defining the purpose and intent of the study through the development of the Operational 
Concept (OC). As defined by Buede (2009), the purpose of the OC is to codify a shared 
vision of the system that characterizes what the system is, what it does, and how it will be 
used. The OC will codify the overarching goals and end-state of the M&S study, and will 
serve to aid the user in maintaining his/her perspective when identifying ESINQ factors 
of interest as well as selection of an appropriate M&S package. While the OC is often 
synonymous with the Use Case, a SE analog typically used to describe the vision for an 
emerging system design, for this work the OC will refer to the operational and synthesis 
models, while the Use Case will be reserved for defining the vision for the physical 
system. Once the operational concept is defined, and the context and boundaries of the 
study established, ESINQ factors of interest can then be identified based on the users’ 
needs and the definitions established during Chapter I. Here, users will identify the 
overarching ESINQ factors of interest to their study. This initial identification should be 
broad in scope, and focused on capturing the essence of the effects in which the user 
wishes to explore, without any consideration for the feasibility of capturing those effects. 
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Once the ESINQ effects of interest are identified, the user will then describe in much 
more detail the expected impacts that these ESINQ factors will have within the context of 
the operational scenario of interest. The detail here is similar to brainstorming, and 
should include any and all potential impacts, regardless of the appropriateness, based on 
the purpose of the study. The intent is to generate a large set of potential impacts to build 
a better understanding of the ESINQ effects prior to identification of tangible effects. The 
end-state of Step 4A is the creation and instantiation of the ESINQ Effects Mapping 
Matrix (EEMM) located in IMDP. This is the primary tool used during the IMDP, and 
serves to translate ESINQ factors of interest to tangible and compatible factors within the 
selected model. With the ESINQ factor of interest identified, and the expected impacts 
codified, we can now link these expected impacts to tangible operational effects. 

Alb: Linking ESINQ Expected Impacts to Tangible Ejfects 

The intent of this sub-step of the IMDP is to refine the understanding of the 
ESINQ impacts gained during Ala, and link them to a set of tangible effects that can 
potentially be used to represent them within the model. The IMDP accomplishes this by 
guiding the user to a better definition of the ESINQ effects of interest through a process 
of brainstorming and subjective assessment. This step builds more detail regarding the 
expected impacts of the selected ESINQ factors, and produces a linked list of potential 
effects that can tie expected impacts to tangible effects within the model. While the user 
focused on establishing the broad interest of the study in Step Ala, the intent here is to 
capture a much more detailed understanding of the tangible effects that can potentially 
represent the ESINQ effects of interest. This is important because we have yet to select 
an M&S package, and the understanding gained here will help screen and select an 
appropriate M&S package for use within the study. Eollowing the identification and 
codification of these tangible effects, the EEMM is expanded to demonstrate the linkages 
between expected impacts and a set of newly determined tangible effects that can 
potentially be used within the model to represent the impacts to operations from the 
ESINQ factors. With a better understanding of the tangible effects which we believe 
could be used to represent the impacts from ESINQ factors within the model, we can now 
review potential M&S packages for appropriateness, and select one for use. 
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Ale: M&S Suitability Assessment and Selection 

The intent of this sub-step of the IMDP is to investigate potential models and 
select one for use that has sufficient flexibility to represent the tangible effects identified 
in Step Alb, which can then be used to act as surrogates for the intangible impacts 
identified during Step Ala. The IMDP accomplishes this by providing the user with a 
formali z ed methodology for assessing the feasibility of potential M&S packages for use 
within a particular study. This methodology, which will from this point on be referred to 
as the MSSSM, can be found in Appendix A. The MSSSM provides the user a the means 
for framing his/her needs of an M&S package, and following this conceptualization, the 
MSSSM moves the user through the review, screening, exploration, assessment, and 
finally, the selection of an appropriate package for use within their study. Though the 
MSSSM is time consuming, it has the potential to greatly increase the accuracy of the 
M&S study by limiting the potential errors introduced through the use of non-optimal 
M&S packages. While the MSSSM is not required to execute the IMDP, if time and 
resources are available it is recommended. 

Following the completion of the MSSSM, a weighted decision matrix will be 
produced that will allow the user to compare the top three M&S packages through a total 
value score based on the needs and weights of the user. For this dissertation, the MSSSM 
was used to select Map Aware Non-uniform Automata (MANA) as the M&S package for 
this research, which was developed by McIntosh et al. (2007) on behalf of the New 
Zealand Defense Technology Agency. While not described in detail here, all 60-i- pages 
of that MSSSM assessment can be found in Appendix A. Following the selection of an 
appropriate M&S package for use within the study, as well as the identification of the 
surrogate factors that can best represent the tangible effects described in Step Alb, the 
EEMM is expanded to demonstrate the mapping between tangible effects and surrogate 
factors within the selected model. This mapping serves two purposes. Eirst it identifies 
which surrogate factors can represent the most tangible effects, which supports weighting 
of factors. Second, the mapping ensures traceability from surrogate factors, through 
tangible effects and expected impacts, back to the original ESINQ factors of interest. 
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The intent of the translation sub-step of the IMDP was to manipulate ESINQ 
factors into a form which could be understood and implemented by the model. 
Translation lead the user through identification of the ESINQ factors of interest, their 
expected impacts linked to a set of tangible effects, the selection of an appropriate M&S 
package, and the identification of surrogate factors to represent the effects of the ESINQ 
factor. Translation left the user with a better understanding of the surrogate factors that 
could be used to represent the impacts from ESINQ factors within the model, as well as 
the selection of an appropriate M&S package based on the needs of the study and the 
availability of surrogates for the ESINQ effects of interest. With this understanding, we 
are ready to move on to the next step of the IMDP, which is Normalization. 

(2) A2: Normalization 

The intent of this sub-step of the IMDP is to normalize the reference tool that will 
be used to calibrate the selected model in the next step. In order to establish an accurate 
baseline for calibration and the subsequent model development, any ESINQ-enabled bias 
in the reference tool must be stripped out before implementing the model development to 
ensure that confounded factors are minimized. Normalization has two primary steps, and 
begins with the selection of an appropriate reference tool, typically one which the users 
use for operational assessments. This can be an assessment tool, as used in this work, or 
another model. After the reference tool is selected it must then be normali z ed to establish 
a base-line by separating out the internal and external sources of combat power. Because 
the modeling of some systems may be based on data that includes combat power derived 
external to the system boundary, it may be required to decouple both internal and external 
effects in order to support a better articulation of the actual sources of combat power. At 
the conclusion of this sub-step, the selected reference tool has been normali z ed and is 
ready for use to calibrate the surrogate effects identified in the translation step. 

Ala: Selection of a Reference Tool 

The intent of this step is to select an appropriate reference tool for use during the 
calibration and validation of the model. While the reference tool is not technically 
required to execute the IMDP, it ensures a more accurate assessment of the ESINQ 
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effects within the model, the analysis derived from that model, as well as provides some 
level of validation, depending on the validity of the reference tool. Executing the IMDP 
without a reference tool would lead to a highly subjective process that cannot be 
guaranteed to improve the accuracy of the model. The IMDP provides a framework that 
supports the selection of a reference tool by providing the user with considerations that 
will help guide them through the screening and selection of an appropriate reference tool 
based on the intended need. This directed exploration of potential tools is much more 
likely to produce an appropriate tool than would have been achieved without the use of 
the IMDP. For this dissertation, the IMDP led to the selection of the Army’s FRC as the 
reference tool for this work, which will be discussed in more detail in the next chapter. 

Alb: Normalization of the Reference Tool 

The intent of this step is to establish an accurate baseline for the calibration and 
development of the model by removing any potential ESINQ-enabled bias from the 
reference tool. Any bias left in the reference tool will be passed on to the model 
following calibration, and thus, degrade the accuracy of the model to represent the 
impacts from the ESINQ factors we are interested in. Thus, the user must inspect the 
source data of the reference tool in detail, and ensure that any such bias is removed to 
ensure that confounded factors are minimized. The source data that drives the FRC is 
based on the COFM and expanded by the Army’s Forces Analysis-System Weights and 
Normalization (FA-SWN) spread sheets, which were obtained from the National Training 
Center (NTC) Fizard Team by request. Unfortunately, each reference tool can have a 
vastly different set of source data, and thus, it is impossible to provide a formali z ed 
methodology for stripping out this bias. To overcome this issue, the IMDP provides a 
general framework and example to guide the user through the identification and removal 
of bias from the source data of the reference tool. Once the bias is removed, we can 
update the source data of the reference tool with these improved values, creating an 
improved FRC that better captures the system level contributions to combat power, 
yielding a better system versus system representation of combat. With normalization of 
the reference tool complete, we can now calibrate Mission, Agent, External Dependence 
(ExtDep), and ESINQ factors in the model. 
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(3) A3: Calibration 


The intent of this step of the IMDP is to incrementally calibrate the surrogate 
ESINQ factors selected during translation of the reference tool to link the outcomes of the 
model to the selected reference tool. Additionally, because the reference tool in this case, 
the Improved Relative Combat Power Assessment Tool (IRCPAT), includes 
dependencies on mission sets as well as system specific attributes; mission and agent 
factors will also need to be calibrated. It is this complexity that makes calibration the 
most complicated step within the IMDP, and depending on the user’s requirements, can 
demand a significant amount of effort. The general flow of the calibration sub-step can be 
seen in Figure 45. 



Figure 45. Calibration Building Blocks 


Calibration begins with baselining activities, where we prepare both the reference 

tool and the model for use. From this point forward, we begin an incremental spiral 

development process where we expand the capabilities and accuracy of the model and the 

reference tool while maintaining their synchronization. The IMDP accomplishes this by 

directing the user through a series of design iterations, and using DOE and statistical 

analysis tools, conduct increasingly detailed design space exploration to establish the 
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mission, agent, external dependencies, and finally the ESINQ attributes for each agent. 
The first of these iterations is Mission Calibration, where the model and reference tool 
are expanded and calibrated to account for the impacts from the selected mission sets on 
Blue Probability of Victory (Pv), which quantifies the defensive advantage. The next 
iteration is Agent Calibration, where the model and reference tool are expanded and 
calibrated to account for the system level (Agent) differences between opposing forces, 
which quantifies the advantages due to system capabilities. The third iteration is ExtDep 
Calibration, where the model and reference tool are expanded and calibrated to account 
for each agent’s dependency on external factors for the generation of internal combat 
power. The final iteration is ESINQ Calibration, where the model and reference tool are 
expanded and calibrated to account for the baseline values of the ESINQ surrogates of 
interest, which will be used in later steps as a reference point for quantifying operations 
impacts. A full description of the steps of calibration will be provided here while 
avoiding as much redundancy as possible. Thus, to avoid the inherent replication of the 
IMDP Calibration steps, it will not be presented linearly as it was intended to be 
executed. Rather, the process will be covered in a single iteration, with the specifics of 
Mission, Agent, ExtDep, and ESINQ calibration being discussed as needed to describe 
the differences between the steps. Eor more detail regarding the complete process, please 
refer to Chapter IV, Demonstration and Analysis as well as in the IMDP Part A3a-e. Eet 
us begin calibration by discussing baselining activities. 

A3a: Baselining Activities 

The purpose of baselining is to establish a foundation for the incremental 
calibration of mission, agent, external dependencies, and ESINQ factors that will follow. 
This step focuses on the preparation and development of the simplified model and 
reference tool needed to begin calibration. Baselining activities has three primary steps, 
first, to instantiate the reference tool, second, to develop and calibrate the simplified 
model to the reference tool, third, to establish the ER attributes to be used in the DDEs. 
Each step will be briefly discussed, starting with the instantiation of the reference tool. 

To instantiated the reference tool, the ERC is modified to establish a baseline for 

calibration, which is a simplified scenario where we will minimize all variables other 
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than the mission set and Foree Ratio, which will be ignored in the baseline. This will 
provide a starting point for quantifying the operational impacts from the mission set on 
metrics of effectiveness, as well as linking the model to the reference tool. Because of the 
future intended purpose of this tool, and its current inability to represent the impacts from 
ESINQ effects, the FRC needed to be expanded and improve in numerous ways to make 
it useable for the purposes of this work. While these improvements and modifications 
will not be discussed here, they will be discussed and demonstrate in detail in Chapter V. 
The improved and expanded FRC, which will from this point forward be called the 
IRCPAT, and can be seen in a partial form in Figure 46. For a full description of the 
IRCPAT refer to Chapter V. 


Improved Relative Combat Power Assessment Tool (IRCPAT) 
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Figure 46. IRCPAT (Baseline) 


As shown in Figure 46, the FRC was significantly modified to create a more 
robust and modern operational planning tool. Here we can see that the IRCPAT was 
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instantiated for the baseline case, which is two identical forces without any 
considerations for Mission, Agent, external dependencies, or ESINQ factors of combat 
power. This is seen in the equal force type and size, the equal mission type, and the 
zeroing out of all ESINQ sources of combat power in the IRCPAT. Thus, as expected, the 
IRCPAT shows a 1:1 Eorce ratio, with an expected Blue Py of 50%. With the reference 
tool instantiated, it is now time to develop the simplified model. 

The simplified model is a small controlled experimental environment that will 
support the analysis of the impacts from factors of interest on forces during calibration. 
The model will provide an abstract representation of the system of interest, accurately 
represent all interaction of the system with the external environment at a level of 
resolution appropriate to produce output data of enough fidelity to support the intended 
analysis. Using the M&S package selected during the execution of the MSSSM, the user 
develops the model to the requirements as outlined in the OC established during part 
Ale. The development of the model does not differ greatly from traditional MDPs other 
than the IMDP directing the user to address three specific considerations. Eirst, the IMDP 
directs the user to use an increased number of agent attributes for later use during the 
calibration steps. Second, the IMDP directs the user to initially limit the number of 
differing agents and fix the majority of all attributes to better highlight the impacts of the 
select factors of interest on Blue and Red forces. Third, the IMDP directs the user to build 
the simplified model as accurately as possible, but without any external factors. This is 
key; all agent attributes that are expected to be used in the complete model must be 
instantiated in the simplified model with logical and defendable agent settings. Tailing to 
establish agent attributes here (i.e., leaving default settings), and then attempting to 
modify the setting value later, after calibration, can have a significant impact on the 
accuracy of the model, and can introduce potential anomalies. 

While the IMDP cannot provide a formali z ed process to do this, which would be 
impossible due to the nearly infinite amount of potential variables and user inputs in to 
the process, it does provide a framework. This framework supports the user’s selection of 
appropriate agent factors for use within the simplified model, as well as a list of key 
points that the user should consider when developing the model. This framework takes 
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the user through agent selection, model development, and then finally model verification, 
the detailed steps of which can be found in the IMDP. For baselining activities, this is a 
fairly straight forward process, requiring the development and verification of a model that 
has two identical sets of forces pitted against each other, without any advantage to either 
side being gained due to differences in Agent, Mission, ExtDep, or ESINQ factors. This 
billiard board modeling environment ignores all aspects of combat other than Eorce 
Ratio, and after verification, should yield a fairly consistent fight where the Py converges 
on 50% for the Blue force when the RCP is 1:1. This should be confirmed through a 1000 
replication verification run prior to moving forward. Eollowing model development, the 
last step is to establishment of Eorce Ratio Attributes. 

The purpose of establishing Eorce Ratio Attributes is to determine the range of 
Blue agents that will be needed to capture the full range of expected victory’s based on 
the users selected mission set and modeling needs. Eor this work, this range would result 
in a Py between 40% to 95%, which, based on the mission set, will help determine the 
number of agents. The IMDP provides a tool for determining these numbers in A3a, and 
using the Hasty Attack - Hasty Defense (HA-HD) mission set and its corresponding 
mission modifier (2.5), as well as the starting Red strength of 20, the tool recommends 
that the DOE should vary the number of Blue tanks from 46 to 83 to explore the full 
range of potential victory cases, with 50 tanks giving the Blue force a Py of 50%. This 
Eorce Ratio factor and ranges will be the primary method to delineating numerical 
superiority in the DOEs used in future model exploration. 

A3b-e: Mission/Agent/ExtDep/ESINQ Calibration 

The intent of calibration is to expand and link the model and the reference tool 
through the establishment of Mission, Agent, ExtDep, and then ESINQ specific attributes 
that will capture the impacts that these attribute have on metrics of operational 
effectiveness within the model and reference tool. This is an iterative process, and when 
repeated during the spiral development of the model to account for Mission, Agent, 
ExtDep, and then finally ESINQ attributes, the process resembles the description seen in 
Eigure 50. 
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Figure 47. Calibration Flow 


As shown in Figure 47, the calibration of all four attribute types (Mission, Agent, 
ExtDep, and ESINQ), is an iterative process, where the IMDP directs the user through the 
set of four primary steps. Eirst, calibration begins with the expansion of the reference tool 
to include combat power contributions from the selected attribute. Second, the attribute 
factors of interest are selected, their ranges estimated, and then the model is expanded 
and updated to include these factors. This step is where the majority of the differences 
between the four iterations reside. Third, a multi-factor DOE is designed and executed to 
produce detailed output data that captures all factor interactions within the model with 
respect to impacts to metrics of combat effectiveness, which include Py, Eorce Exchange 
Ratio (EER), and Eoss Exchange Ratio (EER). Eorth, analysis of the DOE output is then 
conducted in order to establish and record the factor values for the selected attribute 
which results in the least variation of the Blue Py estimated by both the model and the 
reference tool across the full range of potential ERs. Regardless of the type of attribute 
we are attempting to calibrate, the calibration process will always have these same four 
primary steps. In fact, the steps of these four iterations are nearly identical, only differing 
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slightly in the method of calibration for each of the four attribute types. Thus, for brevity, 
the full calibration process will only be shown for a single iteration, highlighting the 
differences for each of the four attribute types. Let us begin with the expansion of the 
reference tool. 

Expansion of the Reference Tool 

The first iterative step is the expansion of the reference tool to account for the 
combat power contributions from each of the four attributes. To expand the reference 
tool, the IRCPAT is modified to include considerations for the selected attribute on Blue 
Py. For Mission calibration, this expansion allows the reference tool to account for the 
impacts of Mission set on metrics of combat effectiveness. This quantifies the advantage 
of the defending force, which for this example was a Hasty Attack versus Hasty Defense. 
For Agent calibration, this expansion allows the reference tool to account for the impacts 
of systems specific capabilities on metrics of combat effectiveness. This identifies and 
accounts for the differences in opposing systems, like the differences between the M1A2 
and the T80U used in our example, which quantifies the advantages gained from superior 
weapons systems. For ExtDep calibration, this expansion allows the model to account for 
the impacts of the external contributions to weapon system internal combat power 
derived from sources outside the system boundary, which are already accounted for in the 
IRCPAT. For ESINQ calibration, this expansion allows the reference tool to account for 
the impacts of ESINQ factors of interest on metrics of combat effectiveness. This initial 
expansion will only capture the baseline (no impact) of all ESINQ factors, and thus will 
not impact either the tool or model outcome at this point, but will be further expanded in 
part B of the IMDP to include the rest of the ESINQ factors of interest. The expanded 
IRCPAT following Mission, Agent, and ExtDep calibration can be seen in Eigure 48. 
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Improved Relative Combat Power Assessment Tool (IRCPAT) 



Figure 48. IRCPAT (Post Calibration) 


As shown in Figure 48, the baseline IRCPAT was modified over three iterations 
(ESINQ calibration does not impact this tool) to account for the impacts of these 
attributes on metrics of combat effectiveness. First, the IRCPAT was modified during 
Mission Calibration, which established the selected mission set, which in this case, had 
the Blue force conducting a Hasty Attack and the Red Forces conducting a Hasty 
Defense. This yielded a significant advantage to the Red Force, who was defending, 
which dropped the Blue Py from 50% to roughly 3%. Thus, to maintain the 50% Py, the 
Blue Force Ratio was increased to 2.5 to 1, which required 50 Blue tanks versus the 20 
Red tanks. Next, the IRCPAT was modified again during Agent Calibration, which took 
into account the differences between systems of the opposing forces, which in this case, 
had the Blue force operating MlA2s and the Red Forces operating TA-80Us. This 
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yielded an advantage to the Blue Force, who was operating a superior weapons system, 
which increased the Force Ratio from 2.5:1 to 2.95:1, and the Blue Pv from 50% to 
roughly 72%. Thus, to maintain the 50% Py and a 2.5:1 Force Ratio, the number of Blue 
tanks was decreased from 50 to 42. Finally, the IRCPAT was modified one last time 
during ExtDep Calibration, which took into account the dependencies of opposing 
systems on external sources for internal combat power. This more accurately assesses the 
systems combat power by including both the systems combat power and external sources 
of combat power. Because the weapon systems of the Blue force (Tier I) derive more of 
their total combat power from external sources than the Red force (Tier II), this yielded 
an advantage to the Blue Force. This increased the Force Ratio from 2.5:1 to 2.79:1, and 
the Blue Py from 50% to roughly 65%. Thus, to maintain the 50% Py and 2.5:1 Force 
Ratio, the number of Blue tanks was decreased again from 42 to 38. With the reference 
tool expanded to account for Mission, Agent, and ExtDep attributes, it is now time to 
execute calibration and model expansion. 

Expansion of the Model 

Eollowing the expansion of the reference tool, the second iterative step focuses on 
identify the attribute factors of interest, estimate the ranges of potential impacts, and then 
expanded and updated the model to include these factors. Because this process is heavily 
dependent on the type of attributes which we are calibrating (Mission, Agent, ExtDep, 
ESINQ), each iteration through this step is slightly different. Thus, of the four primary 
steps of calibration, this is the step where the majority of all differences reside. To ensure 
the reader understands the complexity of this step, each of the four calibration iterations 
will be discussed individually, starting with Mission calibration. 

The IMDP leads the user through Mission calibration by providing the user a set 
of generalized steps that allow for the classification of the range of impacts from the 
surrogate factors being used to represent the impacts of Mission set on metrics of combat 
effectiveness. To do this, the IMDP provides a tool that directs the user through five 
general steps, which include: the establishment of a relative hierarchy of impacts; the 
identification of Mission set values; the estimation of the maximum impacts from these 
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factors; the establishment of the DOE ranges for these Mission factors; and finally the 
expansion of the model. This process is described in Table 5. 


Table 5. IMDP Part A3b: Mission Attributes 


Impact 

Hierarchy 


Mission Set 
Transfer 


Estimated Max 
Values 


O 


Update the 
Model 


2 

Mission Calibration: These are attributes of each agent that are tied to the mission set, these help 
define ‘advantage* of defending forces. Determine the range of the Mission Attributes based on the 
specified mission set of each force by selecting the appropriate input values (Yellow cells) In the 
following table, if you do not wish to use this DOE calculator and already have an understanding of the 
required factor ranges, input them directly into the next table. 

- For this example, the following multiplication factors were used. 

Modifiers 


DA 

HO 

DO 

Ph Combo Mod Max 

PersCooc 

Ph Against 

Ph From 

0.30 

0.40 

0.60 

0.80 

DD-HA 

1.00 

0.12 

1.00 

0.90 

0.60 

0.30 

DO-OA 

0.90 

0.18 

; 0.40 

0.60 

0.90 

1.00 

HD4IA 

0.90 

0.24 

RateOfFire 

; 0.40 

0.60 

; 0.80 

1.00 

HD-OA , 

’*'0.81 

0.36 

Speed 

J 0.60 

0-80 

1 0.00 

OOP 

HA-HA, 

0.40 

0.40 

\ /X 


Transfer these values to\he ’Mod* columi/in the DOE Fac}pr^tting Tool Below. 


- Insert your ’Maximum* va^s (in the for^t accepte^Ay the model for that factor) into the DOE 
Tools based on knowledge a^ expected performadte of the systems being modeled. 

0 Red Hasty D^env ^ 




\|od|^ Max DOE Low Model DOE High 


Blue Concealment 

0.30 

90 

10 

27 

57 


Red Concealment 

0.60 

90 

24 

54 

84 

Blue Ph (at 3500m) 

0.24 

0.40 

0.05 

0.10 

0.30 

RedPh(at 3S00m) 

0.90 

0.40 

0.16 

0.36 

0.40 


Blue RoF (R/lOO) 

0.40 

30 

5 

12 

27 

Red RoF(i»/100) 

0.80 

30 

9 

24 

30 


Blue Speed 

0.60 

48 

10 

29 

48 

Red Speed 

0 

0 

0 

a 

0 




As shown in Table 5, the purpose of this step is to select and establish the range of 
Mission Attributes that will be used to represent the impacts of the Mission set on the 
model. Simply speaking, these are the attributes that will be used to quantify the 
“advantage of the defense” within the model. The four factors shown here were identified 
during the author’s investigation of MANA, and were selected for their ability to 
manipulate model outcomes without inducing significant complexity. These factors 
include Personal Concealment (Peon), Probability of Hit (Ph), both against and from, Rate 
of Fire (RoF), and Speed. While this work recommends that the user of the IMDP use the 
same factors, some modifications or alternatives may be needed based on the selected 
M&S package and needs of the user. To execute this step, the user begins by inputting the 
expected relative impacts of each of the five factors in the four general mission areas. 
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From this, the tool calculates the Ph Combo Mod that accounts for the specific mission 
type combination, of which there are nine possible combinations. The user then transfers 
the specific mission type values of interests (HA-HD for this example) into the DOE Mod 
column, as well as the Ph Combo Mod, and after inputting the maximum estimated value 
each of these factors can have, the min and max values for each of the four factors (Red 
and Blue) are calculated. The last step is to expand the model to account for these new 
factors, and to conduct a verification run to ensure that the model remains synchronized 
with the expanded reference tool. After the model verification, the model is ready for 
DOE and analysis, which will account for the differences between opposing forces based 
on the Mission set, and will be discussed in later. Eollowing Mission calibration, the 
model can now be expanded yet again through Agent calibration. 

The IMDP leads the user through Agent calibration by providing the user a set of 
generalized steps that allow for the identification of new Agent factors as well as 
modification of existing Mission factors to represent the system specific differences 
between opposing forces. To do this, the IMDP provides a tool that directs the user 
through five general steps, which include: system comparison; assessment of the system 
differences; quantification of these differences; establishment of the DOE ranges; and 
finally the expansion of the model. This process can be seen in Table 6. 
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Table 6. IMDP Part A3c: Agent Calibration 


System 

Comparison 

Delta 

Assessment 


Delta 

Quantification 

Establish DOE 
Ranges 

Update the 
Model 


Af«nt Calibration: Calibrat# tha Afant Attribotasfor aach agant to account for variations in 
capabilitias basad on tha Improvad Forca Analysis Systam Waights Spraad shaats to craata tha 
final ‘A^ant Attnbutas' to ba usad in tha Modal. 

• Compara tha systam attributas using tha sourca documants of tha rafaranca tool. 

o Captura tha diffarancas batwaan thasa agants in a tabla. Thasa S araas, and associatad 

1 Systam Attributas (spaad) 

2 Protaction Attributas (Enamy Ph, Concaalmant) 

3 Waapons Attributas (Ph) 

4 Sacondary Waapons Attributas (Ph) 

o Tha complatad assassmant of tha systam diffarancas can ba saan balow. In this 


Attribut* 

WaiKht 

M1A2 

T-«OU 

V-SH 

Modalad? 

Calibrsta? 

Systpm 

0.2S 

0.17 

0.17 

0 . 0 % 

Yas 

No 

ProCACtion 

0.35 

0.38 

0.28 

26.39i 

No 

Yas 

Weapons 

0.35 

0.21 

0.22 

•4.8H 

Yas 

No 

AuxWaap 

0.05 

0.08 

0.05 

37.5H 

No 

No 

Intar C4| 

N/A 

0.25 

0.201 

19.6% 

No 

Yas 

F.E. xlO 

1.00 

10.90 

9.21 





O Nota: Intarnal C4I comas from Stap A2b (Intarnal C4I FE), and accounts for roughly 23H of 
o Datarmina if tha modal naads to ba calibratad to account for tham. If tha systam 
^^^Quanti^Tn^Wnga^^agan^ttriouta^aada^^aprasan^nKysTa^^maranca^^^ 

M1A2 ara 26.3H mora protactad than tha T*60U, so raduca thaT*60s Ph by 26.3H 
M1A2 Aux Waapons ara 37.SH battar. If waapons ara modalad individually, disragard, 
M1A2 has 19.6M battar sansors. If sansor rangasA<f«Pc ara known, disragard, validation 
Modify tha modal to account for thasa systam diffarancas. 

o Instantiata tha tabla balow using tha assassmantsfrom abova. Nota, this is a rough 
o Nota: Whan salacting a choosing individual factors it is somatimas possibla for 



Factor 

Mission 

Actributa 

Cal 

Modifier 

Weight 

Agant 

Actributa 

Nota 


Blua Acant Soaad 

30 



30 


From Stap 

A3b 

(Mission) 


Rad Acant Spaad 

0 

- 

- 

0 


Blua RoF 

12 

- 

. 

0 



Rad RoF 

26 

- 

- 

0 



Blua Concaalmant 

lO 

- 

- 

lO 


Rad Concaalmant 

65 

. 

. 

65 



BluaPh{at3500m) 

0.150 

- 

- 

0.150 


Rad Ph (at 3500m) 

0.400 

0.263 

0.70 

' 0.326 

Main Gun 

Blua Td (4000m) 

8.000 

- 

- 

8.000 


New 

(Agant) 

Rad Td (4000m) 

8.000 

c ; 

0.70 

' 9.098 

Vah 


Blua Pc (pardat) 

0.350 

- 

- 

0.350 



Rad Pc (pardat) 

0.350 

0.196 

0.80 

0.295 

Vah 


^oOfthasafactors^Spaad^RoF^^onc^andPhhavaalraadvbaanmissionealibratadintha 

o Tha FR rangas will naad to ba adjustad to maintain tha 40H to 95H victory calibration 
o *6ast’ rangas (High or low), should ba fixad basad on tha mission attributas abova. I.a., 



DPs 

Factors 

Low (40H) 

Modal 

High (95%) 

FR 

Attribute 

1 

a Blua Tanks 

40 

43 

59 

Mission 

Actributa 

2 

Rad Ph (at 3500m) 

0.326 

0.350 

0.400 

Agant 

3 

Rad Td 

8 

9 

11 

Attribute 

4 

Rad Pc 

0.295 

0.325 

0 350 


The purpose of this step is to select and establish the range of Agent attributes that 
will be used to represent the impacts due to system level differences between weapon 
systems of the opposing forces. Simply speaking, these are the attributes that will be used 
to quantify the advantage of superior firepower within the model. To execute this step, 
the user begins by comparing the source data of the opposing forces system by system, 
and identifying the major areas where data support accurate comparison. For this 
example the source data for the IRCPAT included the comparison of systems in five 
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areas: system attributes, protection attributes, primary weapons attributes, secondary 
weapons attributes, and Command, Control, Communications, Computers, and 
Intelligence (C4I) attributes. Following the comparison, these differences were then 
assessed to determine if they were already being accounted for in the model, and if the 
difference was significant enough to demand calibration of the model. If the answer to 
both these questions was an affirmative, then calibration was deemed necessary. Of the 
10 potential Agent factors identified during system comparison and assessment, only 
three were determined to require calibration, and included Red Ph, Red average Time 
between Detections (Td), and Red Probability of Classification (Pc). The calibration of 
these three factors would be used to represent the system level differences between the 
opposing forces (tanks in this example). Of these three, Ph had already been accounted for 
during model calibration, thus, its value was simply updated. The other two Agent 
factors, Td and Pc, had not been used previously, and therefore, require that both Red and 
Blue forces instantiate these Agent factors in the model. While both must be implemented 
within the model, the Blue Agent factors will remain fixed while the Red Agent factors 
are varied within the DOE. 

Following this assessment, the user quantifies the impact from the system level 
differences, and calculates an estimated value for each of the Agent factors requiring 
calibration. For this example, the baseline factors settings from the model were modified 
by the % difference noted during assessment to achieve a modified version of each of the 
Agent factors, which now roughly takes into account the difference between systems. 
Please note, this is not a true quantification of Agent factors, but rather, it is just a rough 
estimate that only serves to inform the DOE which will follow. After the Agent factors 
are estimated, the potential range of these factors must be established. This is done 
similarly to before, with a mix of suggested ranges provided by the IMDP tools and 
subjective assessment by the user. While a broader DOE range will increase the 
exploration trade-space, it will also reduce the saturation of the solution space, thus 
requiring more M&S replications. Eikewise, too narrow of a DOE window could limit 
the exploration space to a point that the user can risk not capturing a feasible Agent factor 
setting value. Thus, care must be made by the user when bounding DOE ranges. The last 
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step is to expand the model to aeeount for these new faetors, and to eonduet a verifieation 
run to ensure that the model remains synehronized with the expanded referenee tool. 
After the model verifieation, the model is ready for the DOE and analysis, whieh will 
aeeount for the differenees between opposing forees based on the speeifie systems used 
by both forees, and will be diseussed in later. Following Agent ealibration, the model ean 
now be expanded yet again through ExtDep ealibration. 

The IMDP leads the user through ExtDep ealibration by providing the user a set 
of generalized steps that allow for the identifieation of new ExtDep faetors as well as 
modifieation of existing Mission and Agent faetors to represent the agent internal 
dependeneies on external systems. To do this, the IMDP provides a tool that direets the 
user through four general steps, whieh inelude: assessment of the agent differenees; 
quantifieation of these differenees; establishment of the DOE ranges for these ExtDep 
faetors; and finally the expansion of the model. This proeess ean be seen in Table 7. 
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Table 7. IMDP Part A3d: ExtDep Calibration 


Delta 

Assessment 


Delta 

Quantification 


Establish DOE 
Ranges 


Update the 
Model 


External OepeiKferKy Calibratton: Calibrate tbe final ESINQ Attributes to identify th« steady state 
settings for these factors (to lr>clude the contributions from external dependierscies on internal metrics 
of combat power) that cause r>o disruption to the outcomes of the calibrated model and referecen tool. 
These will act as the referecne from which future manipulation of these factrs to implement ENSIQ 
effecst Will be made. 

iMHIilAIM. imiMIMlI MMIII. 


Ixl MiUUitU lULUAI XMli<l 

depersdencies). thus, we have baselined the IRCPAT ar>d model for system level combat power. 
Now we must account for thesystes internal dependencies on external contributions to combat 
power. This is allready done in the IRCPAT. but no we must again calibrate the model to the 
IRCPAT. this time for external dependencies. 

Copy the calibration attributes th ranges during Step A3c here. 

Exa rsded ( lOOH Ext Cow 
HA~ 





Cone 

lO 

65 


Ph 

0.15 

0.345 


RoF 

12 

26 

Spe€d 

30 

0 


Pc 

0.3S0 

0.300 


Td 

8 

lO 


These base llr>e attribute valu< 


inserwh^F^an^Ex^ependerscie: 


1 


will represent the worse case scenario (OH contribution to 
Mh^agen^ein^calibrate^rom Step 4F. 


M1A2 

TA-«OU 




’|for tl 


o The increases to combat powv of both forces will neJd to be accounted for in the operational 
model to represent the contribution from external sArces. Thus, we will need to modify the 
baseline calibration attributes to account for this in^ease in combat power, and then use these 
new values as the lOOH external contributions to injernal combat power. The difference 
between these two sets of valtes will drive the meta'inodels used to represent the impacts 
from internal dependcricics on external sources of gombat power in the operational model. 

^Snt^^^SttnCute^romTBo^^Ta^a^B^us^R^Cpresen^^SontriSotSn^o^terna^^^" 

systems on internal combat power. Because external ^pendencies revolve mostly around C4I. I 
would suggest attributes that woiM likely be impacted by it. like: 

Ph 1 Rea I 


C4i Attributes: I 


T 


X 


Because one force is likely nojto be moving, speed is only partially usefull. 

Because Td is currently set at 8/10 sec respeettv^. it is limited on the low er>d. and thus does not 
have much room for improvement, and thus, is fcelyjust a secondary factor at best. 

Pc has strong ties to C4i. and 1 lot of wiggle room. thus, is a good primary factor for calibration. 
White Ph IS not directly impacted by C4i. it is infrectly. and thus a usable modifier if the other 
factors don't have ersough polrer to make the Required changes, to be safe, include it in the 
OOE for r>ow. but try to minimize changes. ^ 


Transfer these factors in to the folmat below. The tool will produce a DOE range for the calibration 
of external contributions to internal combat powf r. Note, depending on the direction of 
improvement, the high or low of Jach attribute will remained fixed at the baselirve values, this will 
ensure that no improvment will^Suse a reductipiyn capabilities. 


ICt^^l 


Factor 

Agent 

Attribute 

Cat 

Modifier 

ExtDep 

Attribute 

OOE Low 

Model 

DOE High 

Blu* Ph (3500m) 

0.150 

0.187 

0.178 

Q=aso 

0.178 

0.206 

Red Ph (3500m) 

0.345 

0.085 

0.374 

0^45 

0.374 

0.404 

Blue Td (4000m) 

8 

0.187 

6.5Q2 

4 

6 

8 

Red Td (4000m) 

10 

0.085 

9.145 

8 

9 

10 

Blue Pc (per det) 

0.350 

0.187 

0.416 

0.850 

0.416 

0.481 

Red Pc (per det) 

0.300 

0.085 

0.326 

0.300 


0.351 


o The FR ranges will need to be adjusted to maintain the 40H to 95H victory calibration bounding. 
To do this, use the ircpat instanciated at the begging of this step to adjust the number of 
agents (Blue Tanks) in order to identify the number needed to achieve 40H. 50H. 90H. and 9SH 
victory. For this example, this was 36, 38. 49, and S3 tanks. 

un i iiutes. 


The purpose of this step is to select and establish the range of ExtDep attributes 
that will be used to represent the impacts due to system dependencies on external systems 
for the generation of internal combat power. Simply speaking, these are the attributes that 
will be used to quantify the advantage of external dependencies within the model. To 
execute this step, the user begins by comparing the source data of the opposing forces 
system by system, and identifying the difference between opposing forces with regard to 
external dependencies. Eor this example the source data for the IRCPAT focused on the 
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comparison of C4I attributes. Following the assessment and comparison, these 
differences were then quantified, and potential surrogate factors for these differences 
inspected for relevance to the model. Of the six potential ExtDep factors identified during 
system comparison and assessment, only three were determined to be acceptable for 
potential use, and included Red Ph, Td, and Pc. The calibration of these three factors 
would be used to represent the system level dependencies on external systems for the 
generation of internal combat power. Following this assessment, the user inputs the 
values in to the DOE tool to estimate the range of these ExtDep factors within the DOE. 
The last step is to expand the model to account for these new factors, and to conduct a 
verification run to ensure that the model remains synchronized with the expanded 
reference tool. After the model verification, the model is ready for the DOE and analysis, 
which will account for the differences between opposing forces based on the specific 
systems used by both forces, and will be discussed in later. Eollowing ExtDep calibration, 
the model can now be expanded yet again through ESINQ calibration. 

The IMDP leads the user through ESINQ calibration by providing the user a set of 
generalized steps that allow for the identification of new ESINQ factors as well as 
modification of existing Mission, Agent, and ExtDep factors to represent the impacts 
from the ESINQ factors of interest. To do this, the IMDP inserts the ESINQ factors 
identified and bounded in Step Aid (as well as the EEMM) into the calibration process 
by providing a simplified process that directs the user through four general steps, which 
include: ESINQ factor transfer; factor de-confliction; quantification of these differences; 
establishment of the DOE ranges for these ESINQ factors; and finally the expansion of 
the model. This process can be seen in Table 8. 
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Table 8. IMDP Part A3e: ESINQ Calibration 


Transfer ESINQ 
Factors 


De-conflict 

Factors 


Establish DOE 
Ranges 


Update the 
Model 


I ESINQ Calibration; Calibrate the ESINQ Attributes to identify the steady state settings for these 
factors that cause no disruption to the outcon>es of the model and reference tool. These will 
act as the reference from which future manipulation of these factors to implenrvent ENSIQ 
effects will be made in Part 6. 

- Copy the ESINQ Attributes identified from the EEMM Step Ale here. ** For demonstration 
reasons, i added in a few extra ESiNQ effects to fully explain the process. 



Input Factor 

Mission 

Agent 

ExtDep 

Calibrate 

Oeoenden 


Ave Det Time (sec) 


Yes 

Yes 

“So ■ 



Prob Classification (%) 

Yes 

Yes 

No 

Yes 

• a 

Latency (sec) 





No 

aa 

Reliability (H) 





No 

aa 

Ph per Discharge (%) 

Yes 

Yes 


No 

yes 



o Disregard any factors that have already been captured in mission, agent, or ExtDep 
calibration. In this example, this includes; 



Mission Calibration: 

Ph 


Agent Calibration: j 

Ph. Td. Pc 


ExtDep Calibration: 

Td. Pc 


o Assess the system dependency on the ESiNQ effect. For direct fire systems, with 
shooting based on Agent/squad SA. they can see the target and have limited 
dependencies on latency and reliability. For indirect systems, which shoot based on 
inorganic SA. are not as much effected by Td or Speed, but are heavily dependent on 
o If all ESINQ factors are accounted for in either Mission. Agent, or ExtDep calibration, 
or not necessary due to lack of dependency, this step is not necessary, all ESINQ 
factors have already been calibrated for steady state, and the user can move on to the 
o In this example, an M1A2 is a direct fire system, and thus, not sufficiently dependent 
on Latency or reliability, and thus, these factors would not need to be calibrated. To 
keep this demonstration moving forward, lets pretend we are looking at an indirect 

^ansfe^h^emainin^actor^^^h^orma^elov^UnTik^Mission^^nt^n^xtKp 

attributes. ESINQ attributes are currently fixed. The purpose here to simply include them 
into the model without impacting the output (assessing the baseline values for these 
ESINQ values). These values may or may not be mission/agent/ExtDep independent, and 
thus, the baseline values should not be assumed to be equal for both red and blue forces. 


1 

Attribute 

Range 

DOE Low 

Model 

DOE High 


25 

20K 

20 

25 “ 

30 

Red Latency (sec) 

20 

20% 

16 

20 

24 

Blue Reliability (%) 

99 

5% 

94 

99 

100 

Red Reliability (%) 

99 

5% 

90 

95 

100 


o Transfer the expected values from part A3a, then set a confidence range. This is an 
example, the values here are for demonstration purpose only, 
o Note: Because latency and reliability effect external squad communications, in this 
calibration model it will be applied too and from the links with higher HQ. In the 
example, this would not have any impact on the model, because as stated direct fire 
systems are not dependent on external comms. But to continue with the example, 
pretend that we were looking at artillery, who is dependent on external SAfor 
targeting. The model vwuld have included a source for that data, a HQ likely, and it is 
iMi^SlN^actor^>ee^^^aTiorated^ner^^i^?ee^^ontinu^ni^tep^nner^s. 

the user would continue on as before, executing the following steps to conduct a DOE, 
analysis, and verification runs of the model to establish the baseline values of the ESINQ 
factors while ensuring that the model and the reference tool continue to remain 


As shown in Table 8, the purpose here is to import the ESINQ factors (surrogates) 

that were identified in Step Aid, and establish the range of these factors that will ensure 

we can capture the steady-state value of each ESINQ factors during the DOE. Once 

calibrated, these values will serve as the baseline settings for each agent in the model, and 

provide a reference point from which future manipulation and trade-space explorations 

can be compared. This differs from the other calibration steps in a few ways. Eirst, the 

steady state ESINQ factors are independent of the mission, agent, and ExtDep attributes 
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and thus, do not vary based as the others are changed. Second, while the other attributes 
will remain fixed in the model following calibration, the ESINQ attributes will not, and 
will be manipulated in order to demonstrate the impacts from the ESINQ factors. Thus, 
the ESINQ attributes calibrated here are intended to capture just the steady-state value, 
not the range of impacts due to the ESINQ factors; this will take place in part B. 

This step starts with the inclusion of the five ESINQ factors identified during Step 
Aid and codified in the EEMM. Once imported, the surrogate factors that were 
previously addressed during Mission calibration were ignored, which for this example, 
included Ph, RoE, and Speed. Eikewise, any surrogate factor that were addressed during 
Agent and ExtDep calibration were also ignored, which included Red Td. This left just 
three unique ESINQ factors for inclusion. Blue Td, as well as Red/Blue Eatency (Eat). 
The ranges identified for these ESINQ factors in the EEMM are then transferred into the 
DOE format, establishing the ESINQ Attribute Ranges. The last step is to expand the 
model to account for these new factors, and to conduct a verification run to ensure that 
the model remains synchronized with the expanded reference tool. With the calibration 
steps complete, and the reference tool and model expanded to account for all three 
attribute types of interest, it is now time to move on to the next step, DOE, and produce 
the data necessary to support a thorough investigation of the trade-space. 

Design of Experiments (DOE) 

The third iterative step is the selection and build of the DOE to produce the 
required data necessary for a thorough exploration of the trade-space for each of the four 
attribute iterations. The purpose of the DOE is to provide an engineered approach to 
system experimentation specifically designed to produce data efficiently, across the entire 
system trade-space. As Beery (2016) mentions in his work, the “existing MBSE 
methodologies, as well as recent research in MBSE, fail to emphasize the importance of 
proper experimental design selection in the development of external models and 
simulations to support MBSE focused system development” (125). Through the proper 
use of DOE, it is possible for a user to efficiently explore the trade-space of a system, 
producing data of enough depth and detail to ensure that all significant interactions within 

the system as well as between the systems and the OE are identified. 
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The DOE step is identical for all four iterations, and does not change as we 
expand calibration to include Mission, Agent, ExtDep, and finally ESINQ factors. While 
the steps do not change, the focus of each DOE does. Eor Mission calibration, the 
purpose of the DOE is to capture the steady state values for the factors that will represent 
the impact of the specified Mission set on Py. This DOE will likely have the most factors, 
as well as the broadest range of these factors, which will significantly increase the 
dimensionality of the solution space. Thus, this DOE will need to be large enough to 
ensure adequate saturation of the design space through techniques such as design 
stacking. Eor Agent calibration, the purpose of the DOE is to capture the steady state 
values for each Agent that will represent the impacts of systems specific capabilities on 
Py. This DOE will focus on fine tuning known agent characteristics, and thus, will likely 
have fewer factors than Mission calibration as well as a much reduced range of these 
factors. This will reduce the size of the solution space, which will in turn reduce the size 
of the design needed to ensure adequate saturation of the design space. ExtDep 
calibration is very similar to Agent calibration, with the only difference being the factors 
selected to capture the steady state values for the external dependency of each Agent in 
the DOE. Eor ESINQ calibration, the purpose of the DOE is to capture the steady state 
values for each ESINQ factor of interest. Because the intent here is to establish a baseline 
value to serve as a reference point for future inclusion of ESINQ, and not to quantify the 
value, this DOE does not need to be nearly as detailed as the others. This detail will come 
later in the IMDP. Coupled with the fact that this DOE will not need to account for 
ESINQ factors previously calibrated, this DOE will likely have the fewest number of 
factors, which will again reduce the size of both the solution space as well as the design. 

The importance of selecting an appropriate design cannot be overstated. The use 
of modern space-filling designs that ensure a thorough investigation of the trade-space as 
well as minimize the pairwise correlation of factors have vastly increased the 
understanding and insight gained from model exploration. As we saw with the use of 
non-optimal M&S packages, the use of non-optimal DDEs can be just as damaging to the 
development of an accurate understanding of the OE. Thus, users should use the most 
detailed design possible within the intent and constraints of the study. Eor more 
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information regarding DOE, to include a current catalog of modem designs, please refer 
to the NFS Simulations, Experiments, and Efficient Designs (SEED) Center for Data 
Earming website, at https://harvest.nps.edu/ . 

The DOE is executed for each of the four iterations to produce data of sufficient 
resolution to adequately inform the analysis process of each calibration step. The first of 
these iterations was for Mission calibration, which for this example included a 
combination of Eorce Ratio and Mission attributes, leading to nine total factors. Rather 
than selecting a nine factor 2"‘* order NOEH design as the IMDP would suggest, to save 
resources in this example, this work uses a much simpler NOEH design based on the 
work of Cioppa and Eucas (2007). The simpler design resulted in a much smaller design, 
just 33 DPs opposed to the 265 DPs required by a 2"‘^ order NOEH design, and once 
stacked nine times and replicated 100 times, resulted in a 29,700 DP simulation run. 
While this simple design only guarantees near orthogonality for main effects, for this 
demonstration the loss in design resolution is well worth the reduction in complexity. The 
second of these iterations was for Agent calibration, which for this example included a 
combination of Force Ratio and Agent attributes, as well as one of the Mission attributes, 
resulting in a total of four factors. As before, a simpler NOEH design (seven factor) was 
used rather than selecting a four factor 2"‘^ order NOEH design as the IMDP would 
suggest. After stacking and replication, the NOEH design resulted in an 11,900 DP 
simulation run. The third of these iterations was for ExtDep calibration, which for this 
example included a combination of Force Ratio, Mission, Agent, and ExtDep attributes, 
resulting in a total of seven factors. As before, a simpler seven factor NOEH design was 
used for a total of 11,900 DPs. The final DOE iteration was for ESINQ calibration, which 
for this example included a combination of Force Ratio and ESINQ attributes not 
captured in previous calibrations, which reduce the number of ESINQ factors from 10 to 
just four. Using the same designs from the Agent and ExtDep DOEs, this DOE resulted 
in an 11,900 DP simulation run. Following the development of the designs, they were 
uploaded to the NFS advanced computing cluster along with the model, and after the 
output data was returned, analysis was possible. 
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Analysis 

The fourth and final iterative step in calibration is analysis, where the output data 
from each of the four DOE iterations is explored and analyzed to determine the steady- 
state factor values for Mission, Agent, ExtDep, and ESINQ factors. Eike DOE, the 
analysis step is identical for all four interactions and does not change as we expand 
calibration to include Mission, Agent, ExtDep, and finally ESINQ factors. In fact, the 
only difference between the four iterations is the focus of the analysis, specifically the 
end-state. Eor Mission calibration, the end-state is to establish the baseline settings for the 
eight factors (Peon, Ph, RoE, and Speed, for both Blue and Red) that will represent the 
impacts of the selected Mission set on Py. Eor Agent calibration, the end-state is to 
establish the baseline settings for the three factors (Red Ph, Red Td, and Red Pc) that will 
represent the impacts from the assessed differences between the opposing weapon 
systems on the Py. Similar to Agent calibration, ExtDep calibration will use Red Ph, Td, 
and Pc to represent the impacts from the agent’s internal dependencies on external 
systems to generate combat power in support of Py. Eor ESINQ calibration, the end-state 
is to establish the baseline settings for the three factors (Blue Td, Blue Eat, and Red Eat) 
that, in combination with the other seven ESINQ factors already accounted for, will 
represent the impacts from ESINQ factors of interest on the Py. Analysis of the DOE 
results is executed for each of the four iterations to determine the attribute settings for the 
selected mission set within the model. These baseline settings will provide a set of factor 
settings that will capture the impacts of the selected Mission/Agent/ExtDep/ESINQ 
factors on the Py within the model, calibrating the model output to the reference tool {+!- 
5%) across all Eorce Ratios. Additionally, the baseline ESINQ settings for each agent 
will also provide a reference point from which trade-space explorations can be analyzed 
in later steps to quantify the operational impacts from ESINQ factors. 

This step starts with the Analysis and exploration of the DOE data. Eor this work, 
IMP was used to conduct the statistical analysis, regression, and contour profiling needed 
to determine the settings of the Mission, Agent, ExtDep, and ESINQ factors settings. The 
analysis began with a regression analysis to assess the significance of the individual 
factors, two-way interactions, and higher order interactions on the Py. The regression was 
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then used as a screening tool to remove any insignificant factors. Following the 
regression, the contour profiler was used to explore the trade-space of the remainder of 
the factors, as demonstrated in the Mission Analysis example in Figure 49. 
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Figure 49. Contour Profiler from Mission Analysis 


As shown in Figure 49, by setting the maximum variance between the model 
results and the expected results from the IRCPAT (victory curve #4) to +1-5% and then 
conducting TSE, a set of Mission factor settings were identified that ensured consistent 
results between the model and the reference tool, as indicated by the range of feasibility 
(white area) across Force Ratios of 2.3 to 3.65. While these settings do not cover the full 
calibration range of 2.0 to 3.65, the fact that the deviation is on the low side of Py, from 
roughly 25% to 45% Py, most users would be willing to overlook it because any solution 
in this range would likely be considered unacceptable. These factor settings capture the 
Mission specific advantages of the defending force, and are recorded for future use. 
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These factors will be used to update the model prior to the Agent, ExtDep, and ESINQ 
calibration iterations, and following the completion of all four calibration iterations, will 
be used in the development of the complete model. With factor settings recorded, the next 
step is to verify the results by executing a one factor DOE, where we vary just the Eorce 
Ratio (#Blue Tanks) to verify that the output of the model mirrors the expected results 
shown by the reference tool across the full range of potential ERs. The results of this 
analysis can be seen in Eigure 50. 



Eigure 50. Analysis Verification: Mission/Agent/ExtDep Attributes 

The verification DOE confirmed that the Mission factor settings identified 
through the calibration iterations produced nearly identical results between both the 
model and the reference tool when implemented. Thus, the model is Mission calibrated 
with the reference tool and capable of producing results that fall within +/- 5% of the 
reference tool across all ERs. Eollowing mission calibration, the expansion of the model, 
and recording of the Mission settings, the calibration process described in this section is 
repeated for Agent, ExtDep, and ESINQ factors during their specific calibration 
iterations. During these iterations, the Mission analysis results, which captures the model 
factor settings, are iteratively modified and expanded to capture Agent, ExtDep, and 
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ESINQ baseline factor settings for the model. The complete calibration verification can 
be seen in Figure 51. 
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Figure 51. Mission, Agent, ExtDep Calibration Results 


As shown in Figure 51, as the model was expanded to capture a more accurate 
representation of the OF, the RCP of the Blue force increased, which required a reduction 
in the number of Blue Tanks to maintain the same Py range (40%-95%). The end state of 
part A is a calibrated set of Mission, Agent, ExtDep, and ESINQ attributes that accurately 
link the reference tool to the model. When introduced into the model, these attributes will 
enforce consistent outcomes between the reference tool and the model, across all 
potential mission sets and FRs. At this point, these attributes are ready for inclusion into 
the final model and will provide a stable reference point for the assessment of ESINQ 
effects on metrics of combat effectiveness. 

b. Part B: Model Development 

Part B of the IMDP expands the capabilities of current MDPs and MBSE analysis 
methodologies by providing users an improved methodology for producing more accurate 
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models utilizing the calibrated data from part A. The purpose of model development is to 
finalize the implementation of the simplified model as envisioned in the OC, and then to 
expand that model to highlight the impacts that ESINQ factors can have on metrics of 
operational effectiveness. As described in Figure 52, part B has two primary steps. 
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Figure 52. IMDP (Part B) 

By providing a means for models to account for ESINQ factors, the models can 
capture a more accurate understanding of the OF. Part B of the IMDP does this through 
the formali z ed realization and manipulation of the model, with the goal of improving the 
overall utility of the model by increasing the accuracy of the supporting data. A more 
detailed description of part B can be seen in Table 9. 


Table 9. IMDP: Part B 


PartB 


la 

0«velope th« model to improve its capacity to meet user needs and requirements, 
without any modification to the ESINQ factor surrogates from 
Mission/Agent/EictOep/ESiNQ calibration identified in Part A. This is an iterative 
process where you buM model capacity by adding in other modeling aspects ignored 
during calbration, like other forces. Conduct analysis as needed to adjust and fine tune 
the model. 1000 iterations of the model should be calibrated to within (V- SK) of the 
output from the IRCPAT. To verify, execute a 1 x factor DOE to establish a analysis 
reference point for future comparison. 

2a 

Expand the model to include the - '. . factors identified in Part A by modify the 
. . factors from their baseline values to a setting whKh achieves the expected 
impacts from each ' . effect. This task supports the development of operational 
support tods. 

2b 

Execute a full 2** Order NOLH DOE across all Force Ratios to establish a new analysis 
reference point, used to quantify the operational impacts from the ' effects of 
interest across all full range of severity. Same DOE process as described in Part A3b-e. 

2c 

Quantifying ESINQ Effects; Data analysis and Meta.iTiodel development for the 
responses of the ' factors, quantifying operational impacts. Same Analysis 
process as described in Part A3be. 


Realization 


Manipulation 
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Both realization and manipulation can be broken down further into lower level 
processes, but as before, these lower level processes and their associated products will 
not be discussed in depth here, but will be defined and demonstrated in Chapter IV. The 
intent of this section is to provide the reader more detail regarding the conceptual flow 
described in Figure 46, specifically the functions of realization and manipulation. 

(1) Bl: Realization 

As described in Table 9, the purpose of realization is to finalize the model build as 
described in the OC, based on the users’ needs, requirements for the study, the selected 
mission set, without any modification to the ESINQ factors or surrogates. Realization 
focuses on improving the simplified model used in part A through the addition of all 
remaining elements of the model, to include other forces that were ignored during 
calibration. At the conclusion of this step the complete model will be constructed, and it 
will be synchronized with the reference tool and ready for expansion to account for the 
impacts from ESINQ factors if interest. 

Bla: Model Development 

Eortunately, the majority of model development has already been completed, 
resulting in a fairly simplified development process when compared to the development 
of the model in part A3a, and thus, this step will not be discussed in detail, although it 
will be demonstrated in the next chapter. At this point, all aspects of the model that were 
identified as requirements in the OC should have already been implemented in the model, 
though they may not all be active yet. Thus, there are only some small modifications 
necessary to implement the aspects of the full model that were ignored in part A due to 
directives of the IMDP. Eor the most part, these aspects deal with the addition of other 
forces not calibrated during part A. During this step, each of these additional forces is 
integrated into the model through an implicit calibration technique described in the 
IMDP. This is a “partial” calibration from those conducted during part A, and because 
our baseline calibration focused on the most significant system/s, it is possible to add 
additional forces to the model at relatively low resolution while maintain synchronization 
with the IRCPAT by modifying just a few select factors of each agent type. The IMDP 
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directs the implicit calibration of these agents through the execution of a DOE, where the 
DOE ranges are based on the respective percentage of contribution to the overall RCP of 
each agent. Eollowing model development and implicit calibration, verification of the 
model is conducted to enforce linkages with the reference tool. To do this, the model is 
replicated 1000 times to verify that the model is still calibrated to within (+/- 5%) of the 
output from the operation reference tool (the IRCPAT) developed in the previous step. 
Then, a one factor DOE is executed to capture the impacts of changing Eorce Ratio on Py. 
The purpose of this verification is to set the conditions for the future expansion of the 
model to account for the impacts of the ESINQ factors of interest, as well as to collect 
summary statistics to serve as a reference point for the quantification of ESINQ impacts. 

(2) B2: Manipulation 

Manipulation focuses on expanding the model to account for the impacts of the 
ESINQ effects and then, through comparison of the model outcomes (Manipulation), to 
quantify the effects of the ESINQ factors on metrics of combat effectiveness, which is the 
end state of the IMDP. Once the model is verified and analysis complete, the model is 
then expanded to include the users expected impacts from the ESINQ factors of interest. 
Eollowing the development of the model, the results of the reference tool and the model 
will begin to deviate, based on the impacts that the ESINQ factors and their effects had 
on the model. It is this deviation in which we are interested in, and by measuring it will 
provide the quantifiable assessment of the impacts of the ESINQ factors on metrics of 
operational effectiveness. This is done in a manner similar to the process of calibration 
described in part A, where we systematically expand the model, execute a DOE, and then 
conduct analysis to capture the impacts. Eollowing these steps, we use the resulting meta¬ 
models to inform the reference tool, and thus, re-linking it to the model. This expansion 
of the calibration process discussed in Eigure 47 can be seen in Eigure 53. 
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Figure 53. Model Development Expansion of Calibration 


As shown in Figure 53, the Mission, Agent, ExtDep, and ESINQ factor settings 
determined during part A are used to develop the model in part B, which will capture a 
much more realistic representation of the OE than the model used during calibration. 
Once the model is developed, an iterative process is used to expand the model as needed 
by the user. For this work, this included the expansion of the model to include the ESINQ 
factors of interest, and following DOE and detailed statistical analysis of the generated 
data, meta-models can be generated to serve as surrogates for the model. These meta¬ 
models can then in turn be used to support the understanding of the systems within the 
OE, including the impacts from ESINQ factors. The use for these meta-models is nearly 
limitless, and when used in conjunction with other tools, can provide a more refined 
understanding of the system within the OE. Manipulation has three primary steps: 
expansion of the model to account for ESINQ effects; DOE selection and execution; and 
analysis and meta-model development. 
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B2a. Expansion of the Model 

The purpose of this step is to expand the model to account for the ESINQ effects 
of interest identified during part A of the IMDP. While this is a fairly straight forward 
process, requiring only minimal modifications to the model, care must be taken in how 
these modifications are made to ensure this process provides more than a subjective 
assessment. This step of the IMDP guides the user through the formalized bounding of 
ESINQ impacts using a value based process that is informed by quantifiable source data. 
This four step process helps the user better assess and visualize the expected impacts 
from the EISNQ factors of interest. While this impact assessment can be subjective, it is 
more informative if based on data from qualified sources, and thus, should be based to 
the largest extent possible on quantifiable sources of impact data. While this may prove 
difficult, a user who can successfully integrate both the art and science of the MDP, as 
seen in military decision making, will have a much higher likelihood of capturing a more 
accurate range of impacts. Eor this work, the expected impacts based on the SMDC 
assessments of threats and capabilities described in Chapter II and Appendix B were 
used. The ESINQ impact assessment for an example problem can be seen in Table 10. 
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Table 10. IMDP B2a-l-2: Initial ESINQ Response Bounding 


User ESINQ 
Impact 
Assessment 


Visualization 

Tools 


Bounding 
ESINQ Impacts 


Establish DQE 
Factor Ranges 


•l-sl ESINQ Response Assessment 

Import the linked input fectors from part Alc/EEMM into column 1 for the 1st 
ESiNQfactor of interest 

Identify the number of levels of seventy you vrant to represent more is better, and 
will yield a better representation of the eipected response, but for simple curves, it 
may not bee needed This data will drive the creation of response meta-models 
used for instantiating the 00€. Create a Column for each of these, for this eiample. 
I used just S levels of seventy as seen m Column 2 

Input eipected impact (% change from base value) based on severity for each of the 
linked input factors irrto column 2. This should be informed to the largest extent 
possible by Quantifiable sources of impact data The key is to assess as accurately 
as possible the expected form, or shape of the response curve Typically, the form 
of the curve is better understood than the Quantifiable impacts, which is why we 
starthere Thescalln|ofthesecurvesw*ll come later during bounding Usethe 
value curves to help support the impact assessment (the curve). This should be 
assessed by as many $ME as possible, lead mg to an outcome that is acceptable by 
most 
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For other forces, or with other ESiNQ factors, these curves could be Quite 
differertt it is for this reason that these curves should be done for each agent- 
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Because the user will likely only haves basic approximation of the expected 
impacts from E$iNQ factors which is why they are ESINQ. an mvestiMt'on into the 
power of the response to impact operational effectiveness is needed to bound the 
resportse to a region of feasibility. Using the values from above is extremely 
subjective, a best guess, and would likely induce more variability into the model 
outcomes than would be desired This step helps refine the estimation of factor 
settings by bounding them through DOE 

First select a base ESiNQ impact value for each factor To do this, make an 
assessment of the I ikely maximum % impact that the CSiNQ effect would have 
compared to the contributions from External dependencies This grounds the 
assessment of ESiNQ impacts on something that has already been accounted for 
and calibrated m the model 
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Red Pc I RedTd 
r-0Q14 


Import the baseline ESiNQfactor settings from part AJd. to ir«lude the 

modification to account for lOOK external dependencies import just the ESiNQ 
effects identified in Alc/EEMM. using the values from part A3d Recall, a 
degradation to Blue forces in this example is shown through a improvement to Red 
capabilities 



Build a multifactor DOC across the range of impacts established above, using the 
FR range from Bta. Round where necessary in the direction of an expanded range 
For a better model fit apply the range to both sides of the base value, failure to do 
this risks a flawed fit which can induce anomalies mtodata analysis 
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As shown in Table 10, the first two steps of the model expansion directs the user 
through a somewhat subjective, yet informed process that supports the estimation of 
ESINQ impacts of model factors identified in previous steps. The key to this step is to 
supplement this assessment with quantifiable support data that justifies the impact 
assessments made here. The more data you have supporting the impacts assessments 
made here, the more accurate the model will be in representing the ESINQ factors of 
interest. Thus, depending on the user as well as the data available, this assessment can 
span the range from purely quantitative, which is unlikely due to the classification of 
these factors as ESINQ, to purely subjective, with is far less desirable. Even if no data is 
available, the use of SMEs can greatly aid in these assessments, and should be used 
whenever possible. Next, because the user will likely only have a basic approximation of 
the expected impacts from ESINQ factors, an investigation into the power of the response 
to impact operational effectiveness is needed to bound the response to a region of 
feasibility. Using the values from the initial assessment can be extremely subjective, and 
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potentially induce more variability into the model outcomes than would be desired. By 
bounding the response to the model, specifically the impacts from external dependencies 
which were previously calibrated, and then establishing a maximum impact of those 
effects with respect to external dependencies, we can further refine and bound the 
estimated response range. Then, through the use of a small multi-factor DOE, we can 
explore this estimated ESINQ response region and identify the ESINQ surrogate factor 
settings that produce the maximum expected impact while minimizing variability. 
Einally, using this calibrated maximum effect, fit it to the value curve developed earlier, 
and build a representative meta-model of the expected response. This final step, 
following the DOE and analysis can be seen in Table 11. 


Table 11. IMDP B2a-4: ESINQ Response Meta-Model Development 


Fittingthe 
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Visualization 

Tools 


Developing 

Response 

Meta-Models 


4 Modify baseline model fKtorscQinfs 
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The meta-models developed in this step will calculate the factor settings for each 
ESINQ surrogate factor based on the expected level of ESINQ severity, bounded by the 
estimated and calibrated response range established in the previous steps. If desired, these 
meta-models can be used to expand the model by including the impacts from the ESINQ 
factors of interest, bounded by the impact ranges selected in Table 10. This expansion 
requires the modification of pre-existing factor settings to act as surrogate factors for the 
ESINQ factors of interest. Eollowing analysis, this should provide the user the expected 
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impacts to combat effectiveness from the ESINQ effects for the specified mission set and 
task organization modeled. 

B2b: DOE Development and Execution 

This step focuses with the selection, design, and execution of the ESINQ impact 
DOE, which will determine the richness of the interactions between the ESINQ effects on 
the outcomes of the model. This step will follow the same general process as the DOEs 
conducted in part A: calibration, with the two major exceptions being that the factors 
being manipulated here are the EISNQ factors of interest bounded during the previous 
step, and the methods in which the design is built. This process has five general steps as 
described in the IMDP. The first step requires the building of the base design, which will 
include just two of the four total factors, the ER factor (#BlueTanks) and the severity 
factor. A larger than necessary design is typically preferable to ensure saturation, and 
should be used when possible. Eor this four factor design, a seven factor 2"“^ order NOEH 
with 125 DPs would have been recommended. Next, if more saturation of the solution 
space/design space is needed, the user can choose to stack the design. Eor this example, 
the design would have been stacked seven times; resulting in an 875 DP design, 
hollowing the instantiation and stacking of the design, it is then expanded to include the 
remaining two surrogate factors (Red Td and Pc), which have their factor settings 
calculated based on the level of severity at each DP using the meta-models developed in 
part B2a. Next, the design is scoped, where the design is formatted in preparation for its 
execution by the model. This is model specific, and may require scaling, translation, and 
other modification to the input data. Einally, after replicating each DP, 400 times in this 
example, the design resulted in a 350,000 DP simulation run, which was executed on the 
NPS advanced computing cluster. Once the output data was returned, it was possible to 
conduct analysis to quantify the impacts from ESINQ factors on the model. 

B2c: Analysis and Meta-Model Development 

Once the DOE data is generated, it is then analyzed with the purpose of establish 
a mathematical representation (meta-model) of the operational impacts from the ESINQ 
factors of interest on the primary metrics for combat effectiveness, namely Py, EER, and 
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FER. Recall, that the contribution of this work is to provide the means (the IMDP) to 
create an ESINQ enable model, which was technically complete following part B2a. At 
this point we are simply trying to quantify that impact in terms of a meta-model that can 
be used to better inform decision makers as well as for future use in other support tools. 
To do this, a meta-model is developed of the ESINQ factor response on the model (Pv), 
across all potential ERs and each level of ESINQ impact severity identified by the user. 
This is done in the same manner described previously, and simply requires regression and 
statistical analysis using IMP. The meta-models generated through this process, while not 
linked to the system of interest yet (Chapter V), can mathematically quantify the impacts 
of these ESINQ factors of interest on the model. By developing individual meta-models 
now, that tie individual ESINQ impacts to metrics of operational effectiveness, we can 
set the conditions for the future application of the IMDP outcomes in support of 
operational and acquisitions planning. This is useful when considering the inability of 
most tools to account for ESINQ effects, to include the reference tool used in this work. 
Because we enforcing the calibration and linkage of the model and reference tool 
throughout the execution of the IMDP, it is possible to take this meta-model and use it to 
expand the reference tool to allow it to account for ESINQ effects, which is one of the 
secondary contributions of this work. And while the expansion of external tools is an 
application, and technically outside of the scope of the IMDP and this work, it will be 
discussed in Chapter V, specifically with regard to the expansion of the IRCPAT and 
acquisitions TSE. 

The end state of part B of the IMDP is the development of a set of meta-models 
that can accurately represent the impacts of the ESINQ effects of interest on metrics of 
combat effectiveness measured from inside the model. These meta-models are the key to 
expanding current operational and acquisition decision support tools, and will enable a 
more complete understanding of the system and the OE, which presents some significant 
advantages for users. Eirst and foremost, it allows users to quantify the impacts from 
ESINQ effects, which are often ignored in traditional MDPs. Second, it encourages users 
to apply that knowledge to other tools to improve their accuracy. By following the IMDP, 
the user can ensure better synchronization between the modeled environment and the 
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actual OE, improving the overall MDP referent while providing traceability back to the 
originating requirements. Additionally, the IMDP supports some level of validation of the 
resulting outcomes through calibration with a reference tool. 

4. Summary 

The purpose of this chapter was to describe the IMDP and how it expands current 
MDPs and MBSE Analysis Methodologies to more accurately account for ESINQ factors 
within a model. The overarching goal of this work is to improve the decision-making 
process of leaders in both the operational and acquisitions communities by providing a 
more robust analysis methodology that can be leveraged to develop more accurate 
decision support tools. The IMDP described in this chapter provides analysts the means 
to simultaneously evaluate an emerging system’s performance across all four referents of 
D3SOE mitigation. Specifically, the IMDP will allow for the more accurate 
representation of ESINQ factors by bounding their response ranges within models. The 
IMDP presented in this work provides users the ability to loosely quantify or “bound” the 
impacts from ESINQ factors on operational effectiveness, and facilitates a more complete 
understanding of a system’s performance with respect to the OE. Additionally, this work 
supports the development of a range of new and improved operational and acquisitions 
decision support tools that can more accurately account for the impacts of not only the 
effects internal to the system boundary, but also the ESINQ effects on the system. By 
following the IMDP presented here, a better evaluation of the OE can be made, which 
will allow for more informed operational and acquisition decisions regarding the 
allocation of resources. A description of how this work expands the current body of 
knowledge can be seen in Eigure 54. 
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Improving MBSE through 
the developmentand 
application of DOE for use 
in trade-space exploration 
during system conceptual 
design 

MacCalman (2013) 


Improving MBSE and trade- 
space exploration through 
the formalized application 
and linkage of SysML 
architecture products with 
external M&S 


Beery (2016) 


Expansion of MBSE 
Analysis 
Methodologies 


Improving MBSE and trade- 
space exploration through 
the formalized application 
of ESINQ factors into M&S 
to improve decision 
support tools 


Pugsley (2017) 


Figure 54. Expansion of MBSE Methodologies. 

Adapted from Beery (2016) and MacCalman (2013). 


As shown in Eigure 54, this dissertation expands the TSE work of both 
MacCalman and Beery and provides a formal methodology for developing ESINQ enable 
models, which can provide more robust models, that capture a more accurate 
representation of the OE. Once used in conjunction with MDPs and MBSE analysis 
methodologies, this expanded model could provide a more detailed description of the 
system and support a better investigation of the system trade-space. With the description 
of the IMDP complete, we will now move on to providing a more detailed demonstration 
of the complete process using a more current and impactful problem, specifically tied to 
the author’s research interests in ground combat operations in a D3SOE. 
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IV. IMDP DEMONSTRATION 


To highlight the utility of this work, this chapter presents a fully executed IMDP 
to provide context and articulate in greater detail the IMDP described in the previous 
chapter. While the mock scenario and models used in this chapter will all be fairly 
simple, the associated products of the IMDP and the outcomes described here will fully 
demonstrate the overall applicability of the IMDP to quantify the impacts of ESINQ 
effects of interest. The quantification of these ESINQ factors, and the application of this 
knowledge to improved decision making are the root of this work, filling known gaps and 
expanding the current body of knowledge with regard to MDPs. To demonstrate this 
process, this chapter will begin with a description of the scenario, which includes a need 
statement that will drive the IMDP. While simplistic, this mock scenario will provide the 
foundation for the execution of the IMDP that will comprise the majority of this chapter. 
The end state of this chapter will be the quantification of the ESINQ factor and effect of 
interest and its codification through the use of meta-models. 

A. THE SCENARIO 

One of the key enablers to modern operations is the use of UAVs to supplement 
ISR collection activities. Unfortunately, while UAVs have provided immense value to the 
Army, they often lack access, adequate range, and survivability in large-scale conflict, 
especially when considering the threats posed by China and Russia. To mitigate potential 
risks, the Army has been considering the procurement of SmallSat ISR collection 
platforms to expand its collection capacity to other domains like space. Unfortunately, 
before accurate assessments of the overall utility of these systems can be made, the 
dependency of the ground force on space-based collection assets as well as their 
vulnerability to degradation from adversary counter-space activities must be understood. 
With these considerations in mind, the following operational need was used to drive the 
IMDP demonstrated in this chapter. 

The Army is interested in accessing the potential impacts to combat operations 
from adversary degradation of space-based ISR assets. As an introductory exploration of 
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this topic, the Army wants to look at the impacts this degradation can have on a typical 
Armor engagement between the M1A2 Main Battle Tank and a near peer equivalent, like 
the T-80U. To maintain consistency with the threats, mission sets, and the expected OE 
outlined in the AOC, the Army wants to focus on operations at the BN/BDE level of 
warfare, specifically short period engagements where U.S. forces are attacking defended 
positions and at a distinct disadvantage from the adversary forces. Specifically, the Army 
is interested in determining how much more force structure (number of tanks) would be 
needed to overcome the adversary’s counter-space activities. 

By quantifying the dependency of ground forces on space-based ISR assets in 
terms of their contributions to RCP, it will be possible to better understand the OE and 
thus, make better decisions regarding the allocation of resources. This improved 
understanding could provide operational planners with a more accurate assessment of 
adversary counter-space activities, allowing them to more accurately assess the number 
of tanks needed to achieve a desired end state. Eikewise, this improved understanding 
could better support acquisitions by providing quantifiable impact data, enabling decision 
makers to make more informed decisions regarding the utility of competing systems with 
respect to the systems contributions to RCP. It is this overall need that will drive the 
IMDP demonstrated in this chapter. 

B. THE IMDP 

The overall goal of the IMDP is to improve the overall accuracy of the model 
through the directed manipulation of the supporting data. To do this, the IMDP expands 
the capabilities of current MDPs and MBSE analysis methodologies by formalizing the 
process for bounding ESINQ effects in a manner that more accurately translates to 
models, thus increasing the accuracy of the model without increasing its complexity. The 
IMDP does so through the formalized translation, normalization, and calibration of 
modeling input factors to develop a more complete understanding of the OE. Once this 
improved understanding is gained, the part B of the IMDP transfers this improved 
understanding to the model through a formalized process of realization and manipulation. 
A detailed description of the complete IMDP can be seen in Table 12. 
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Table 12. IMDP 


Part A 


Part B 



la 

List and describe the primary external, and seemingly intangible and non-<)uantifiable 
effects in vrhich you are trying to represent. (Use EEMM) 



lb 

Identify and list the tangible impacts on combat operations which the effects in Step A 
would likely have. [Update EEMM) 


Using the MSSSM, investigate potential Combat Models and identify a Model with 
appropriate factors which can best model the tangible impacts from Step 46, that can 
be used as surrogates for the intangible effects. For the rest of this methodology I v/ill 
be using MANA as my M&S package. [Update EEMM) 


Ic 



2a 

Selection of a reference tool 


2b 

Base Line the Force Analysis System Weights Spread sheets to account for C4i 
contributions to combat power, then break this contribution into both external and 
internal contributions [If using the Improved Relative Combat Power Assessment Tool 
(IRCPAT), skip this step, this has already been done. 

3a 

Baseline Calibration. Build a simplified model and instantiate the reference tool. 


3b 

Calibrate the 'Mission Attributes' to establish the baseline mission specific attributes 
for each agent for the selected rrxKtet and mission set. 

3c 

Calibrate the 'Agent Attributes' for each agent to account for variations in 
capabilities based on the improved Force Analysis System We^hts Spread sheets to 
create the final 'Agent Attributes' to be used in the Model. 

3d 

Calibrate the 'External Dependency Attributes' for each agent to account for internal 
dependencies on external sources for the generation of internal combat power to 
create the final 'ExtDep Attributes' to be used in the Model. 

3e 

Calibrate the 'ESiNQ Attributes' to establish the baseline ESINQ specific attributes for 
each agent for the model. 


la 

Develope the model to improve its capacity to meet user needs and requirements. 

without any nvxlification to the ESiNQ factor surrogates from 
Mission/Agent/ExtDep/ESlNQ calibration identified in Part A. This is an iterative 
process where you build model capacity by adding in other modeling aspects ignored 
during calbration, like other forces. Conduct analysis as needed to adjust and fine tune 
the model. 1000 iterations of the model should be calibrated to v/ithin {*/• 59i) of the 
output from the IRCPAT. To verify, execute a 1 x factor DOE to establish a analysis 
reference point for future companson. 


2a 

Expand the model to include the ET NO factors identified in Part A by modify the 

ESiNQ factors from their baseline values to a setting which achieves the expected 
impacts from each:' . effect. This task supports the development of operational 
support tools. 


2b 

Execute a full 2*^ Order NOLH DOE across all Force Ratios to establish a new analysis 
reference point, used to quantify the operational impacts from the ■'. effects of 
interest across all full range of severity. Same DOE process as described in Part A3b-e. 

2c 

Quantifying ESINQ Effects: Data analysis and Meta-model development for the 
responses of the ■ ^ factors, quantifying operational impacts. Same Analysis 
process as described in Part A3b-e. 


Translation 


Normalization 


Calibration 


Realization 


Manipulation 


Using the scenario and the associated need statements from the previous section, 
this section will execute the full IMDP, from start to finish, to demonstrate the utility of 
the process to bound ESINQ effects of interest. To minimize any repetition with previous 
chapters, this chapter is presented in a manner which can best replicate the execution of 
the process from the user’s perspective. It is assumed that the reader has a general 
understanding of the IMDP as outlined in Chapter III and can follow along with the logic 
presented here. Thus, this chapter will forgo the majority of definition and focus on 
demonstrating the development of the IMDP outcomes necessary to progress the user 
through the IMDP and toward the quantification of his/her ESINQ factor/effect of 
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interest. Because the IMDP was developed with the user in mind, it is difficult to 
demonstrate passively, thus, this chapter will be presented from the first person view to 
provide a more realistic demonstration. While the first person is more informal than 
typically desired, it will provide the necessary context to fully articulate the IMDP. 

1. Part A: Model Definition 

Part A of the IMDP expands the capabilities of current MDPs and MBSE analysis 
methodologies by formalizing the process for bounding ESINQ effects in a manner which 
more accurately translates to models than traditional MDPs allow, thus increasing the 
capacity of the model to capture a more accurate representation of the OE without 
increasing the models complexity. The IMDP does this through the formalized 
translation, normalization, and calibration of modeling input factors with the goal of 
improving the overall accuracy of the model through a directed manipulation of the 
supporting data. These 10 sub parts, executed to address the scenario described at the 
begging of this chapter, will now be described. 

a. Part Ala: Identify ESINQ Effects and Expected Impacts 

The intent of this step of the IMDP is to build an understanding of the ESINQ 
effects the user is trying to implement in the model, as well as the expected impacts of 
these effects. The IMDP accomplishes this goal through a simplified process that walks 
the user through the initial intent of the overall study through the definition of the ESINQ 
effects of interest. This process can be seen in Table 13. 


Table 13. IMDP: Part Ala 


1 

Develop the Operational Concept 


- What is the purpose of this study? What is it you are trying to get the model to do? 

2 

Identify the ESINQ effects are you trying to represent in your model 


- List the ESINQ effects 


- Are these Internal, External, or both? 

3 

Describe the impacts of these ESINQ effects 

4 

Record findings in the EEMM 
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As shown in Table 13, this step has just four sub-steps to facilitate an expanded 
understanding of the ESINQ factors of interest and begins with the user defining the 
purpose and intent of the study through the development of the OC. While the OC can 
have many forms, in terms of modern MDPs and this research, the OC should address the 
questions described in Table 14. 


Table 14. Typical OC for M&S 


1 

What are you trying to investigate? 

2 

What type of study are you attempting? (Live/Virtual/Constructive) 

3 

What is the purpose of the study? (Descriptive/Prescriptive/Predictive) 

4 

What are the desired Factors, Responses, MOPs, MOEs? 

5 

What level of the model hierarchy do you want? 

6 

Deterministic vs. Stochastic? 

7 

What level of resolution is needed? 

8 

What are the requirements for WA? 

9 

Time-Step vs. Discrete Event? 

10 

What are your analysis requirements? What results are you looking for? 


While the OC is a foundational document, and thus fairly detailed, for the 
purposes of this demonstration a simplified OC will be more than adequate. To set the 
conditions for the execution of the IMDP, the following OC will be used to support the 
scenario described in part A. 

1. The overall purpose of this investigation is to quantify the impacts from 
adversary space-based ISR degradation on the RCP of friendly forces. 
Thus, I want to quantify the impacts to combat effectiveness with and 
without degradation of space-based ISR within the context of a D3SOE. 

2. Because of the significant number of simulation runs required to fully 
explore the tradespace, as well as the limited time and resources available, 
I will only be considering constructive simulations. While live and Virtual 
M&S tools have potential merits with respect to my work, they have 
resource requirements beyond my capacity as a student researcher. 

3. This will be a mixed methods study in which I will be interested in both 
the descriptive nature of the model to highlight the potential impacts from 
a D3SOE, as well as the predictive nature of the model to allow 
comparison of potential alternatives to develop more robust solutions. 

4. The primary evaluation metric for this study will be measures of combat 
effectiveness. Thus, I will need an M&S package capable of taking 
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combat related mission and noise input factors, and in return provide 
combat related responses from the perspective of ground forces, like Py, 
casualties, length of battle, communications, and shooter-target info. 

5. I am looking for a ground combat model that can accurately model BN 
and BDE level operations, to include reliance’s on reach back support 
from space. Thus, a mission level model is most appropriate. 

6. Because I am interested in combat, and that combat is by nature inherently 
chaotic, a stochastic model is preferred. 

7. Because this is a proof of concept, and time and resources are limited, I 
will be looking for a relatively low resolution model. While not as 
accurate in its representation of the OE, it will meet the intent of this 
research and can be improved in the future with a higher fidelity model. 

8. Because this is a proof of concept, where I am attempting to develop a 
methodology and tools for supporting operational and acquisitions 
decisions, a fully VV&A model is not necessary; face/peer model 
validation should more than meet my need. 

9. Through the execution of the MSSSM, primarily for simplicity, ease of 
use, access, and support, a time-step model was selected for my work. 

10. With regard to analysis, I am interested in the output of the simulation, 
specifically how a given set of input factors affect the output response of 
combat effectiveness. I will also be interested in the behaviors of the M&S 
as it progresses. So, I will need an package that produces outputs 
throughout execution, to include summary statistics at the conclusion. 

Eollowing my codification of the OC, where the context and boundaries of the 
study are established, ESINQ factors of interest can be identified. The initial 
identification of ESINQ effects should focus on capturing the essence of the effect, 
without any consideration for the feasibility of capturing or representing those effects. 
Eor this demonstration, the ESINQ effect of interest to me was the effect of ISR 
degradation on combat operations. Next, I describe in as much detail as possible the 
expected impacts of ISR degradation on combat operations, a summary of which can be 
seen in Table 15. 
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Table 15. ESINQ Expected Impacts 


3 

Describe the impacts of these ESINQ effects 


- Impacts to SA due to degradation of ISR collection capacity. 


0 Reduced SA accuracy, less complete COP. 


0 Reduced availability and decreased timeliness of current intel. 


0 Slower fulfillment of ISR requests. 


0 Decreased speed of battle and responsiveness. 


0 Increased uncertainty regarding understanding of friendly and enemy activities. 


0 Decreased availability of ISR collection assets. 


0 Impacts increase at higher echelons and as the duration of degradation increases. 


0 Larger dependency on other, less capable means of ISR. 


0 Increased uncertainty of locational and targeting data, friendly and enemy. 


The detail here is similar to brainstorming and should include any and all 
potential impacts, regardless of the appropriateness, based on the purpose of the study as 
outlined in the OC. The intent is to generate a large set of potential impacts to build a 
better understanding of the ESINQ effects prior to identification of tangible effects and 
the selection of the M&S package. Eollowing the description and consolidation of the 
expected impacts, the EEMM is instantiated to translate ESINQ factors of interest into 
tangible and compatible factors within the selected model. Table 16 shows the 
instantiation of the EEMM for the scenario in this chapter. 


Table 16. EEMM: Part A (Degraded ISR) 


ESINQ Effect 

Expected Impacts 

Degraded 

ISR 

Reduced sensing capabilities and periodic sensor blinding 

Reduced pace of battle. Decreased SA 

Reduced collection capacity/decrease in capabilities 




As shown in Table 16, degraded ISR is the ESINQ factor of interest in this 
example, and I have instantiated the EEMM by listing a few expected impacts from 
degraded ISR on combat operations. While this example is simplified, and lacks what I 
would consider a sufficient amount of detail to fully describe the impacts of degraded 
ISR on actual combat operations, it is sufficient for the purposes of this demonstration. 
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With the ESINQ factor of interest identified, and the expected impacts codified, I can 
now link these expected impacts to tangible operational effects. 

b. Part Alb: Linking ESINQ and Tangible Effects 

The intent of this step of the IMDP is to refine the understanding of the ESINQ 
impacts gained during Ala by linking them to a set of tangible effects that can potentially 
be used to represent them within the model. The IMDP continues the process introduced 
in step Ala, leading the user to a better definition of the ESINQ effects of interest 
through a process of brainstorming and informed subjective assessment, which can be 
seen in Table 17. 


Table 17. IMDP: Part Alb 


1 

Link ESINQ Expected Impacts to Tangible Effects. 


- 

For each ESINQ factor: 



List all potential tangible effects that the expected impacts listed in part Ala would 

0 

have on your model. 

2 

Record findings in the EEMM. 


As shown in Table 17, this step focuses on refining the understanding of the 
ESINQ effect by producing a more detailed list that links the potential effects from part 
Ala to tangible effects which can potentially be used to represent the ESINQ effects in 
the model. Table 18 provides a simplified list of tangible effects that could potentially be 
used to represent the impacts of degraded ISR within the model. 
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Table 18. ESINQ Tangible Effects 


1 

Impacts to SA due to degradation of ISR collection capacity 


- 

Reduced SA accuracy, less complete COP 



0 

Delays in operations, decreases in Ph, increased Frat, slower pace of battle 


- 

Reduced availability and decreased timeliness of current intel 



0 

Reduced number of PiRs can be addressed, less intel generated, data is older, 
targetable intel is reduced 


- 

slower fulfillment of ISR requests 



0 

Less responsive execution of ISR requests, less ISR available, slows pace of battle, 
forces Cmdrs. to source ISR requirements to internal and less capable systems. 


- 

Decreased speed of battle and responsiveness 



0 

Slower pace of operations, forces friendly forces to be more reactive than proactive. 


- 

Increased uncertainty regarding understanding of friendly and enemy activities 



0 

Increased intel error of enemy locations, dispositions, strength, systems, ect, coupled 
with a slower and less complete understanding knowledge of friendly and enemy 
interactions 


- 

Decreased availability of ISR collection assets 



0 

Less collection capacity, prioritization of ISR requests increases intel delays, lower 
echelon forces receive less actionable intel. 


- 

Impacts increase at higher echelons and as the duration of degradation increases. 



0 Strategic ISR requirements begin to take priority over tactical ISR requirements in 
order to meet national requirements, significantly reducing the ISR collection 
capabilities of the operational force, decreasing pace of operations, SA, and causing 
Larger dependency on other, less capable means of ISR 



0 

Lack of actionable intel and accurate SA forces CMDRs to source operational ISR 
requirements to local sources like UAVs, and ground recon teams, which are far less 
capable and take much longer to execute, thus drastically delaying the intel building 
associated with combat planning and execution. 


- 

Increased uncertainty of locational and targeting data, friendly and enemy 



0 

Decreased speed of the targeting cycle as it becomes more difficult to obtain 
targetable intel, coupled with a decrease in Ph due to the combination of locational 
uncertainty and delay in intel gathering 


By linking expected impacts to potential tangible effects, I am able to build a 
better understanding of the impacts of degraded ISR on the model, which is important 
because the understanding gained here will help screen and select an appropriate M&S 
package for use within the study. Eollowing the identification and codification of these 
tangible effects, the EEMM is further expanded to demonstrate the linkages between 
expected impacts and tangible effects, which can be seen in Table 19. 
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Table 19. EEMM: Part B (ISR Degradation) 


Expected Impacts 

Linking 

Tangible Effects 

Reduced sensing capabilities and periodic sensor blinding 


Slo^ver speed, more recon, decreased SA, decreased Pd/Pc of Ext systems 

Reduced pace of battle, Decreased SA 


Slo^ver speed, increased LOB, increased Advantage to Defender 

Reduced collection capacity / decrease in capabilities 


Decreased Ph of indirect fires, decreased Pd/Pc of Ext systems 


As shown in Table 19, the IMDP supports the identification of a list of tangible 
effects that can potentially be used within a model to represent the impacts to operations 
from degraded ISR. My next step was to take this list of tangible effects, and use it to 
drive my search, assessment, and selection of an appropriate M&S package for my study. 

c. Part Ale: M&S Suitability Assessment and Selection 

The intent of this step is to investigate and assess potential models for suitability 
and then select one for my study. My intent here was to two fold. I was looking for an 
M&S package with enough resolution to represent as many of the tangible effects 
identified in Alb as possible, as well as one with enough flexibility to meet as many of 
the primary and secondary considerations identified in OC. Eor this assessment, I used 
the MSSSM provided by the IMDP, the basic steps of which can be seen in Table 20. 


Table 20. IMDP: Part Ale (MSSSM) 


Invt 

tan 

jstigate Models and identify a Model with appropriate factors which can best model the 
zible impacts from Step B. that can be used as surrogates for the intangible effects. 

A 

Develop the M&S Operational Concept. 

B 

M&S Review and Screening. 

C 

Initial Screening. 

D 

Secondary Screening. 

E 

Model Exploration (for each remaining M&S package). 

F 

M&S Comparison and Evaluation. 

G 

Conclusion and recommendation. 

H 

Application (EEMM). 


The MSSSM provided me with a formalized methodology for assessing my needs 
of an M&S package, and then guiding me through the review, screening, exploration, 
assessment, and finally, the selection of an appropriate package. Because of the 
complexity of MSSSM, it is not described in detail here, but all 60+ pages can be found 
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in Appendix A. Following the completion of the MSSSM, I constructed a weighted 
decision matrix to compare the top three M&S packages as seen in Table 21. 


Table 21. M&S Weighted Selection Matrix 


Weight 

Factors 

MANA 

JDAFS 

SEAS 

6 

Ability to Simulate a D3SOE 

3 

2 

5 

5 

Support Availability 

5 

4 

2 

5 

Cluster Access 

5 

4 

2 

5 

Ease of Use 

5 

2 

3 

4 

Rapid Development 

4 

2 

2 

4 

Existing Models 

5 

2 

2 

4 

Ease of Analysis 

4 

3 

3 

4 

Output Data Density 

4 

3 

4 

3 

Learning Curve 

4 

2 

4 

3 

Behavior Monitoring 

4 

2 

3 

3 

DOE Tools Available 

5 

3 

2 

2 

Ease of DOE Execution 

5 

2 

2 

Total Value 

210 

127 

140 


As shown in Table 21, my execution of the MSSSM for my dissertation research 
identified three potential M&S packages. The first was MANA, a time-step and agent 
based ground combat model. The second was JDAFS, a discrete-event modeling 
framework. The third was SEAS, an air and space time-step modeling framework. Based 
on the weights and factors that I established through the execution of the MSSSM, which 
included considerations for my needs and resources, MANA achieved the highest value 
score. Thus, MANA was used throughout my research as well as this demonstration. 
Following the selection of MANA as my M&S package, I identified the MANA 
surrogate factors that could best represent the tangible effects described in Alb and then 
expanded the EEMM, which can be seen in Table 22. 
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Table 22. EEMM: Part C (ISR Degradation) 


/ / // 

Tangible Effects 

Mapping 

lANA Potential Surrogate Faao 

Slovver speed, more recon, decreased SA. decreased Pd/Pc of Ext systems 


Agent Speed 

2 

Slower speed, increased LOB, increased Advantage to Defender 


Ph / Discharge 

1 

Decreased Ph of indirect fires, decreased Pd/Pc of Ext systems 

AveTime/Det 

3 


Pc 

2 


Rate of Fire 

1 


The expanded EEMM shows the mapping of tangible effects to a set of surrogate 
factors that can be used to represent the impacts from degraded ISR that I identified 
within MANA during the execution of the MSSSM. This mapping serves two primary 
purposes. Eirst, it identifies the surrogate factors that can represent the most tangible 
effects, which can support weighting of factors for inclusion in the model. Second, the 
mapping ensures traceability from surrogate factors, through tangible effects and 
expected impacts, back to the original ESINQ factors of interest. The intent of the last 
three steps was to manipulate the effect of degraded ISR into a form which could be 
understood and implemented by the model. 

d. Part Ala: Selection of a Reference Tool 

The intent of this step of the IMDP is to select an appropriate reference tool for 
use during the calibration of the model. While the IMDP does not provide a formalized 
process here, it did provide me a framework to navigate numerous key considerations 
prior to selecting the reference tool. This directed exploration of potential tools is much 
more likely to produce an appropriate tool than would have been achieved otherwise. The 
general process of this step can be seen in Table 23. 
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Table 23. IMDP:PartA2a 


Select an appropriate reference tool 

• 

The reference tool will support the calibration of the operational model, as well as a 
validation method for linking the expected outcomes of the tool and the model. 

• 

It is important to consider the intent of the study and the desired outcome before 
choosing an appropriate reference tool. 


The following considerations should be made with regard to the reference tool. These 
serve as a framework to guide the user through a set of key considerations that should be 
addressed prior to the selection of a reference tool: 


0 Consider the OC of the selected model, the tool should be similar nature. 


0 The tool can be anything from a data spread sheet, a user tool, or another model. 


0 If the intent is for the model to inform another tool, then consider that tool. 


0 The reference tool should include an accessible set of source data. 


0 The tool should be comparable in size, scope, and resolution to the model. 


0 The tool should facilitate validation of the model. 


0 The tool should have a set of outputs that are in common with the model. 


0 The tool should be expandable once informed by the model. 


0 The tool should provide utility to the user outside of its linkages to the model. 


This framework supported my selection of an appropriate reference tool by 
leading me through a set of questions that forced me to scope and screen potential tools 
based on intended need. For this dissertation, as well as for this demonstration, the IMDP 
led me to the selection of the Army’s FRC. This tool, originally designed in 1999 by 
Major J. Craig during his CGSC course work, is currently the most commonly used 
operational planning tool for accessing the combat power of opposing forces. After 
considering its purpose, its commonalities with combat models, as well as my interest in 
expanding it to account for the impacts from ESINQ factors, the FRC is more than 
appropriate for use as my reference tool. The FRC, instantiated for a scenario similar to 
what was introduced in part A, can be seen in Figure 55. 
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Force Ratios 


Friendly Forces 


Enemy Forces 






Type 


Armor Bn (44 x M1A2) 


1 30 


1 30 


Tank Bn (MIB 40xT80) 


1 00 


1 00 


Friendly Force Equivalent 


1.30 


Enemy Force Equivalent 


1.00 


Ratio of Friendly to Enemy 
1.30:1 


Ratio of Enemy to Friendly 
0.77:1 


Hasty Attack 
15% 


<- Mission -> 

<- Est. Losses -> 




2 

Relative 
Combat 1 

Power 

1 

0 

1.30 


1 00 




Friendly Enemy 

Historical minimum planning ratios. 

Fnendly mission 

FnenOly Enemy 

Position 

Delay 

1 ; 6 


Defend 

1 : 3 

Prepared or fortified 

Defend 

1 ; 2 5 

Hastv 

Attack 

3 : 1 

Prepared or fortified 

Attack 

2 5 ; 1 

Hasty 

Counterattack 

1 : 1 

Flank 


Hasty Defense 
15% 


Instructions: 

1 Select type of unit from drop down list 

2. Input number and type. If less than a whole 

unit use fractions (e g 1 Btry = .33 Bns). 

3. Use comparison of force ratios (graph) and 

historical planning ratios 

4 To calculate damage to each unit, select th( 
friendly and enemy mission from the list 

Remember: Relative force ratios do NOT 
necessarily indicate the chance for success 
for either force! 


Figure 55. FRC (Tank versus Tank) 


The FRC was instantiated to compare the RCP of two opposing tank battalions, 
which is the focus of the example we are using in this demonstration. What one should 
notice immediately is that the tool does not address the impacts from ESINQ effects in 
any meaningful way. This is ok, because while it cannot account for the impacts from 
degraded ISR, it does have many similarities with combat models like MANA, which can 
be linked and calibrated to our selected model through metrics of combat effectiveness. 
Additionally, the source data that drives the FRC is based on the Army’s COFM and FA- 
SWN spread sheets. These documents are detailed enough to support the identification 
and segregation of internal and external sources of combat power, and thus, the FRC is an 
excellent reference tool, and its source data will be used for my normalization efforts. 
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e. Part A2b: Normalization of the Reference Tool 

The intent of this step is to establish an accurate baseline for the calibration and 
development of the model by removing any potential ESINQ-enabled bias from the 
reference tool. With regard to the FRC, this bias was identified after inspection of the 
source data; where I discovered that the FA-SWN did not delineate between internal and 
external sources of combat power derived from C4I, one of the five primary contributors 
to a system’s overall combat power. Fikewise, the FA-SWN was inconsistent in how it 
assessed a system’s C4I contributions to combat power, to the extent that some systems 
accounted for it and others did not. Thus, any exploration of the potential impacts of 
FSINQ factors on C4I could lead to inaccuracies in the expected outcomes of the model 
due to its inability to distinguish between sources of C4I derived combat power. The 
framework in Table 24 provided an efficient means for removing the bias identified in the 
FA-SWN by separating internal and external sources of C4I derived combat power, 
assessing and tracking the individual contributions of C4I based on system tier. 
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Table 24. IMDP: Part A2b 


start with a clean version of the Force Analysis Systems Weights Spreadsheets. 


Delete any C4I contributions to relative combat power. 


Calculate the new FE, include C4l contributions, of all system based on the system Tier level by 


- 

Tier 1 

1.60 




- 

Tier II 

1.40 


F.E. New F.E. 


- 

Tier III 

1.25 

MIA2 

8.35 13.36 


- 

Tier IV 

1.10 




Calculate the C4I contribution of each system by subtracting the base FE from the new FE. 
Below is an example for the M1A2 

F.E. New F.E. C4I 

MIA2 ' 


8.35 


13.36 


5.01 


Separate the Internal and External C4I contributions based on the Tier of the system based on 
the values you select below. For example, 50% of C4I capabilities of a Tier I system are derived 
external of the system. 


- 

Tier 1 

50% 

•• Internal C4I factors will stay in the combat model as "sensor 
and SA attributes". 

- 

Tier II 

30% 

- 

Tier III 

20% 

•• External C4I factors will be accounted for separately on the 

IRCPAT. 

- 

Tier IV 

10% 




C4I Internal External 




M1A2 5.010 r"2l05“I 2.505 






Update the Force Analysis Systems Weights Spreadsheets with the new FE (including internal 





F.E. External 





M1A2 

10.855 

2.505 









Copy these two values for each system into the data tab of the IRCPAT. 


The M1A2 used in my example was normalized following seven general steps. 
First, the C4I contribution was removed, which removed the observed inconsistency of 
C4I contributions from system to system. Second, a new Force Equivalent (FE) was 
calculated based on its tier level. The total C4I contribution was then calculated from the 
difference and the contributions from both internal and external C4I was calculated based 
on the system’s tier level. Finally, the internal element of the C4I was added back into the 
system’s total FE, and the remaining external C4I contribution was tracked separately, 
which served to effectively strip out the observed bias due to external dependencies. With 
the bias removed, or more appropriately, reallocated and accounted for separately, the 
updated EE for the M1A2 could now be used to modify the EA-SWN, a portion of which 
can be seen in Table 25. 
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Table 25. Improved FA-SWN FRC (Tank versus Tank). 

Adapted from (U.S. Army 2004). 



Main Battle Tasks 



835 

1 

1 60 


5 01 

050 


2 50 



7 19 

2 

1 40 

10 07 

2 88 

030 

0 863 

2.01 

9 204 






Captured tn 

Improved 

Relative 

combat 

power 

Assessment 

HIB needs 

lobe 
imparted 
into the 
model 

sensors 

Captured 

in 

IfT^roved 

Relative 

combat 

power 


The FA-SWN was updated for the two types of tanks used in this demonstration 
to remove any bias due to the dependeneies of C4I on external contributors of combat 
power. For my work, this update decreased the internal combat power of the U.S. tanks 
by shifting some combat power to external contributors and increased the overall combat 
power of adversary tanks by giving them credit for external elements of combat power 
that were previously ignored. Once updated, the FA-SWN provides the user with a more 
accurate assessment of the combat power of opposing forces, which clearly delineates 
between internal and external sources of combat power. For example, after taking these 
normalized tank values and inserting them back in to the reference tool, the FRC yielded 
a RCP estimate of 1.18:1, a decrease from the 1.30:1 seen in Figure 58 prior to the 
normalization. This difference is expected when considering the bias that was included in 
the original FA-SWN and highlights how normalization can help better articulate it. 


f. Part A3a: Baseline Calibration 


The purpose of baseline calibration is to establish a foundation for the incremental 
calibration of mission, agent, external dependencies, and ESINQ factors. The general 
process for baseline calibration can be seen in Table 26. 
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Table 26. IMDP:PartA3a 


1 

Instantiate the reference tool. 


- Rank order the contributors of combat power from highest to lowest. 


- Instantiate the reference tool for the most significant of the agents above, as Blue and Red, 


with a Force ratio 1:1,100% strength, and 0% External C4I or ESINQ factors. 

2 

Build a simplified model using the agent type noted above. 


- Use a neutral battlefield where no advantages are gained. 

- Opposing agents should have identical system attributes based on the a system of interest. 


- Perform incremental development of the model to ensure a 50% victory rate (+/- 2%). 

3 

Force Ratio Attributes: Determine the range of the Force Ratios based on the mission set. 


The first step, instantiated of the reference tool, focuses on modifying the 
IRCPAT to establish a baseline for calibration, which provides a starting point for linking 
the model to the reference tool. To do this, I instantiated the IRCPAT as closely as 
possible to the operational scenario based on the OC. Once complete, I rank ordered the 
contributors of combat power from highest to lowest, and identify the top 2-3 agents that 
contribute the majority of combat power. Of these, I selected the agent type with the 
fewest number of agents. The reason I want to use the most significant of the agents is 
because the calibration steps that follow will produce more accurate outcomes. For this 
demonstration, I only have two agent types, and thus the M1A2, which is more capable 
than the T-80U, will be used. Next, a new IRCPAT is instantiated using just the MIA2, 
while reducing the complexity by removing all variables that can differentiate a system 
advantage, which included FR, mission, agent, external dependencies, and ESINQ 
variables. The updated IRCPAT can be seen in Figure 56. 
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Improved Relative Combat Power Assessment Tool (IRCPAT) 


Ratio of Friendly to Enemy 


Enemy Forces 


Mjmlw S»Hh9fA 




F.E FE Tc>(* ExiHMi Esf To** 




F.E TotA EjF&fMf 


100% Armor IM1A2) 


10 85 217.00 2 50 50.00 


100 % 


Armor (Ml A21 


10.85 


21700 


2.50 


50.00 


System Force Equivalent 


217.00 


System Force Equivalent 


217.00 


External Force Equivalent 


50.00 


External Force Equivalent 


50.00 


Status of External C4I 


0.00 


Status of External C4I 


ESIMQ Effects 


ESINQ Effects 


Status of 6R DeoraOation 


External Systems 


External Systems 


SmaKat (SR) 


Friendly Force Equivalent 


217.00 


Enemy Force Equivalent 


217.00 


Ratio of Friendly to Enemy 
1 000:1 


Ratio of Enemy to Friendly 
1 000 1 


Hasty Attack 

<- Mission -> 

Hasty Attack 

40 1 202% 

<- Estimated Losses -> 

201% 1 4 0 

0492 

<- Estimated % Victory (Curve 1 ) -> 


0.508 

0498 

<- Estimated % Victory (Curve 2) -> 


0 502 

0.508 

<- Estimated % Victory (Curve 3) -> 


0 492 

0 504 

<- Estimated % Victory (Curve 4) -> 


0496 


Figure 56. IRCPAT (Baseline) 


As shown in Figure 56, the IRCPAT was instantiated using the MIA2 for both 
forces, without any advantage to either side due to force size, type, strength, mission set, 
C4I, ESINQ effects, or external dependencies. As expected with two identical forces, the 
IRCPAT assessed a RCP of 1:1, with an expected Blue Py of roughly 50%. While the 
IRCPAT gives four potential victory curves for the user to choose from. Victory Curve 4 
will be the only one used in this dissertation. Following instantiation of the IRCPAT, I 
can now develop the simplified model. 

The intent of model development during this step is to build a small controlled 
experimental environment that will support the analysis of the impacts from factors of 
interest on equivalent forces during calibration. For model development, the IMDP is 
similar to traditional MDPs, with the only significant difference being the ability of the 
IMDP to account for the expansion of the model to account for Mission, Agent, ExtDep, 
and ESINQ factors. Using MANA, I constructed a simplified model where two sets of 
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identical forces were pitted against each other, and like the instantiated IRCPAT, would 
ignore all advantages of either side due to force size, type, strength, mission set, C4I, 
ESINQ effects, or dependencies. Despite the lack of detail regarding model development, 
this step was very time consuming, and required me to perform a significant amount of 
T&E to verify that the model was implemented correctly. Eigure 57 is a screen shot of 
my MAN A model. 



Eigure 57. Simplified MANA Model (Degraded ISR) 

As shown in Eigure 57, the model consists of 20 Blue tanks (M1A2) facing 20 
Red tanks (M1A2). All aspects of the expected OE as outlined in the OC are 
implemented within the model, other than the ESINQ effects of interest and any other 
ESINQ effect. The tanks of both sides have identical factor settings, personalities, and 
goals and thus, the execution of the model should produce a near draw between the 
forces. To verify, I executed a 1000 replication verification run of the model and the 
summary statistics can be seen in Eigure 58. 
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" BlueWin 






^ Quantiles 


4 ■» Summary Statistics 



0 



100.0% maximum 

1 

Mean 

0.516 






99.5% 

97.5% 


1 

Std Dev 

Std Err Mean 

0.499994 

0.0158112 



484, 48?^ 

5lfl, 52*. 



1 







90.0% 


1 

Upper 95% Mean 

0.547027 







75.0% 

quartile 

1 

Lower 95% Mean 

0.484973 







50.0% 

median 

1 

N 

1000 







25.0% 

quartile 

0 



c 

' 




10.0% 

0 







2.5% 


0 







0.5% 


0 







0.0% 

minimum 

0 




Figure 58. Summary Statistics (1000 Replications of Simplified Model) 


The verification run of my model resulted in the Blue force achieving a Py of 
51.6%. The variation from the true mean of 50% is well within +/- 2% as suggested by 
the IMDP, and is easily accounted for by the stochastic variation of the MANA model 
due to the number of forces. Additionally, the 95% Confidence Interval (Cl) shows a Cl 
of 0.485 to 0.547, which captures the true mean of 50%. Thus, the verification run 
confirms that the model was implemented correctly and behaving as expected, and thus, I 
was able to move on to establishing the FR attributes. 

The last step before calibration was to determine the FR attributes needed for my 
study. To do this, I used the tool provided by the IMDP to estimate the number of Blue 
forces needed, which can be seen in Table 27. 
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Table 27. IMDP: Part A3a-3 


3 

Force Ratio Attributes: These are required to support the functionality of the reference tool. 
Determine the range of the agent Force Ratios based on the specified mission set by fixing the 
size of the red force in the following table. 

For this example, a FIA-FIO, doctrine states that a FR of 2.5 to 1 will result in approximately a 
50% chance of victory, thus: 

For a FIA-FID scenario with red fixed at 20, and Victory Range of 40% to 95%, we vary 
blue forces from 46 to 83. 








Ret 

Forces are fixed at: 

20 












Victory 

^ Curve #3 (IRCPAT) 







40% 

50% 

95% 





B vs. R 

Modifier 

0.92 


1.65 





DD-HA 

0.29 

5 

6 

10 





DD-DA 

0.33 

6 

7 

12 


Low 

High 


HD-HA 

0.4 

7 

8 

14 

AvsD 

36 

116 


HD-DA 

0.5 

9 

10 

17 

DvsA 

5 

17 


HA-HA 

1 

18 

20 

33 

AvsA 

18 

33 


DA-HD 

2 

36 

40 

66 





HA-HD 

2.5 

46 

50 

83 




DA-DD 

3 

55 

60 

99 




HA-DD 

3.5 

64 

70 

116 




The purpose here is to establish the range of Blue agents that will be needed to 
capture the full range of expected victories, which in this demonstration runs from 40% 
to 95%. To use the tool, I started my inputting the starting Red strength (20), as well as 
the victory range of interest for my study, which for me was a Blue Py of 40-95%. Based 
on these values as well as the mission set of interest to me (HA-HD), the tool shows that 
the DOE will need to vary the number of Blue tanks from 46 to 83 to explore the full 
range of potential victory cases, with 50 Tanks giving the Blue force a Py of 50%. With 
baselining activities complete, I can now begin calibration. 


g. Part A3b: Mission Calibration 

The intent of this step is to link the model to the IRCPAT through the 
establishment of mission specific attributes that can capture the advantage that the Red 
force has in defense while maintaining the synchronization of the expected victory 
between the tool and the model. Table 28 outlines the steps of mission calibration. 
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Table 28. IMDP: Part A3b 


1 

Update the reference tool by modifying it for the selected mission set and Force Ratio while maintaining 
a 50% expected victory. 

2 

Mission Calibration: Determine the range of the Mission Attributes based on the specified mission set of 
each force to define the "advantage" of the defending force. 

3 

DOE: Execute a multi-factor DOE of the calibration model by modifiying Force Ratio (#BlueTanks) and 
verify that the updated model continues to have a good fit with the expected outcomes predicted by 
the reference tool across all potential Force Ratios. 

4 

Analysis: Conduct analysis to establish the baseline mission factor settings. 


Trade-space exploration to minimizing variation form expected outcomes. 


Record the Calibrated Mission Attributes as factor settings for use in the operational model. 


I began mission calibration by expanding the IRCPAT to account for the specific 
mission of each force, which for this demonstration, was the Blue force executing a hasty 
attack and the Red force conducting a hasty defense. Following this modification, the 
number for Blue tanks was modified to maintain the expected Blue Py at 50%. The 
expanded IRCPAT can be seen in Figure 59. 


Improved Relative Combat Power Assessment Tool (IRCPAT) 


Ratio of Friendly to Enemy 
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Friendly Force Equivalent 542.50 

Enemy Force Equivalent 217.00 

Ratio of Friendly to Enemy 

2.500:1 

Ratio of Enemy to Friendly 

0.400:1 


Hasty Attack 

<- Mission -> 

Hasty Defense 

8.7 1 17 4% 

<- Estimated Losses -> 

34.9% 1 7.0 

0.492 

<- Estimated % Victory (Curve 1) -> 

0.508 

0498 

<- Estimated % Victory (Curve 2) -> 

0502 

0.508 

<- Estimated % Victory (Curve 3) -> 

0492 

0.504 

<- Estimated % Victory (Curve 4) -> 

0496 


Figure 59. IRCPAT (Mission Calibration) 
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The IRCPAT has been updated to account for the mission set of the opposing 
forces. Thus, a significant shift in the RCP was seen, reducing the expected Blue Pv from 
50%, to less than 5%, which was expected when considering the significant advantage of 
the defense. To overcome this advantage, I was forced to increase the FR of the Blue 
force to 2.5:1 (or 50 tanks) to maintain a Blue Py of 50%. Following the expansion of the 
IRCPAT, my next objective was to attempt to represent this same advantage in the model 
through mission attribute calibration; the steps of which can be seen in Table 29. 


Table 29. IMDP: Part A3b-2 


2 Mission Calibration: These are attributes of each agent that are tied to the mission set, these help 
define "advantage" of defending forces. Determine the range of the Mission Attributes based on the 
specified mission set of each force by selecting the appropriate input values (Yellow cells) in the 
following table. If you do not wish to use this DOE calculator and already have an understanding of the 
required factor ranges, input them directly into the next table. 

- Accounting for Misson Set Advantage: Because the baseline model gives no advantage for the 
deffense, some considerations need to be addressed to give the defending force an advantage over 
the attacking force. This is meant to capture the majority of the advantage, and is a brute force 
manipulation of potnetial outcomes. The misison calibration steps that follow will serve to "fine 
tune" this manipulation in order to achieve a stable outcome across all possible force ratios. 

0 Set cover for the defending force as follows: 

Hasty Defense: 60% Deliberate Defense: 75% 

- For MANA this requires the addition of a terrain feature to denote cover. 

- If this is not enough, another option is to reduce the Om (Max) Ph of the attacker 

Attacker: 0.75 Defender: 0.95 

- For this example, the following multiplication factors were used. 


Modifiers 

HA 

DA 

HD 

DD 

Ph Combo Mod 

Pers Cone 

0.30 

0.40 

0.60 

0.80 

DD-HA 

1.00 

0.12 

Ph Against 

1.00 

0.90 

0.60 

0.30 

DD-DA 

0.90 

0.18 

Ph From 

0.40 

0.60 

0.90 

1.00 

HD-HA 

0.90 

0.24 

RateOfFire 

0.40 

0.60 

0.80 

1.00 

HD-DA 

0.81 

0.36 

Speed 

0.60 

0.80 

0.00 

0.00 

HA-HA 

0.40 

0.40 


- Transfer these values to the "Mod" columns in the DOE Factor Setting Tool Below. 

- Insert your "Maximum" values (in the format accepted by the model for that factor) into the DOE 
Tools based on knowledge and expected performance of the systems being modeled. 

0 Red Hasty Defense 


Mod Max DOE Low Model DOE High 


Blue Concealment 

0.30 

90 

10 

27 

57 

Red Concealment 

0.60 

90 

24 

54 

84 

Blue Ph (at 3500m) 

0.24 

0.40 

0.05 

0.10 

0.30 

Red Ph (at 3500m) 

0.90 

0.40 

0.16 

0.36 

0.40 

Blue RoF (ft/100) 

0.40 

30 

5 

12 

27 

Red RoF (#/100) 

0.80 

30 

9 

24 

30 

Blue Speed 

0.60 

48 

10 

29 

48 

Red Speed 

0 

0 

0 

0 

0 
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The IMDP provides a framework and tools for addressing the selection of 
potential mission attributes in two distinct ways. First, it established a few broad mission 
attributes that can best represent the advantage of the defending force seen in the 
IRCPAT. For this demonstration, I gave the Red force 60% cover and reduced the Om Ph 
of the Blue force to 0.75. This adjustment effectively gave the Red force a significant 
advantage over the Blue force that can be attributed to the advantage of the defense. 
Next, to give me the ability to fine tune the response (Py) of the model during the DOE, I 
used the five factors recommended by the IMDP and inputted my expected multiplication 
factors for each of the five factors for each of the four general mission areas. For 
example, I assigned a force conducting a hasty attack a concealment value of 0.30, while 
giving the defending force (hasty) a concealment value of 0.60. I did this for all 
combinations, resulting in a logical relative hierarchy that characterized the advantages of 
one mission set over another. Following my assessment, I transferred the specific mission 
values and the Ph Combo Mod (HA-HD) into the DOE range tool, and after inputting the 
maximum value for each of the attributes, the DOE min and max values for each of the 
four factors (Red and Blue) were calculated. Eollowing the establishment of the DOE 
factors and ranges, the design was constructed as described in Table 30. 


177 



Table 30. IMDP: Part A3b-3 


3 DOE: Execute a multi-factor DOE of the updated model by varying Rjrce Ratio and any Agent attribute 
identified above in order to verify that the updated model continues to have a good fit with the 
expected outcomes predicted by the reference tool across all potential Force Ratios after agent 
calibration. 

- Set your Design Points based on the work from above, modifying just the Mission attributes noted 
above. Ensure you provide adequate trade space to explore each factor setting. 


0 stack 1 ( vs. Defense). 




DPs Factors 

Low High 




FR 

Attributes 

1 

# Blue Tanks 

46 

83 




Mission 

Attributes 

2 

Blue Concealment 

5 

50 



3 

Red Concealment 

30 

90 



4 

Blue Ph (at 3500m) 

0.05 

0.25 




5 

Red Ph (at 3500m) 

0.20 

0.60 




6 

Blue RoF (#/100) 

5 

20 



7 

Red RoF(#/100) 

15 

40 




8 

Blue Speed 

10 

48 








Select and Build your Design. 


0 

Determine the number of Factors (8 in this example). 



0 If the number of factors is 13 or less, use a 2nd Order NOLH design. If it is greater than 13 up to 
29, use a NOLH design. Both the NOLHdesigns.xIsx and the MacCalman-2ndOrderNOLH design 
tools can be found at https: //harvest.nps.edu/. 

0 Download the DOE tool and input the values from the previous step. 

0 Consider conducting a screening design to determine significance, this may allow you to reduce 
the number of factors, and thus, use a more detailed 2nd order design in subsequent analysis. 

- Stack the design. 

0 Stack the design to achieve better resolution within the design space as well as the solution 
space, which will provide better fitting models. Remember to vary the columns when stacking. 

- Execute the DOE. 

0 For my work I provided the following files to the NPS SEED Center in order to run my model on 
the advanced computing cluster. 

1 MANA Model .xml 

2 The Design developed using the DOE above (translation to the .csv is needed) 

3 The Study .xml script (this executes the model and design on the cluster) 

- The SEED center can assist in preparing these documents for execution if needed. _ 


The purpose of this step is to select and build an appropriate DOE to support the 
identification of steady state values for the factors that will represent the impact of the 
mission set on Py. For this demonstration, rather than selecting a nine-factor order 
NOLH design as the IMDP suggests, I chose to use a much simpler NOLH design to save 
resources. The simpler design resulted in a much smaller design, just 33 DPs opposed to 
the 265 DPs required by a 2"‘^ order NOLH design, and once stacked nine times and 
replicated 100 times, resulted in a 29,700 DP simulation run. This design, the model, and 
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study execution file were then uploaded to the NPS advanced computing cluster, and the 
output data was returned for analysis; the steps of which are explained in Table 31. 


Table 31. IMDP: Part A3b-4 


4 


Conduct analysis of the DOE output in order to establish the Mission Attributes for the HA-HD mission 
which results in the least variation between the model and the IRCPAT expected Blue Victory 96 across 
the full range of potential force ratios. 

- For this work, I used JMP to conduct regression analysis, and then used the contour profilers to 
identify Mission settings for each mission set that kept variation of the mean less than +/-S% from 
the expected victory based on victory curve #4. 

- Ranges for analysis should be based on the mission set. Below is the suggested Force Ratio Ranges 
for each mission set, both bounding and calibration, based on victory curve #4 of the IRCPAT. 





Force Ratios for % Victor 

i 






Mission Set 

Bound Low 
(40%) 

Calibrate 
Low (50%) 

Calibrate 
High (90%) 

Bound 
High (95%) 

Force with 
Speed = 0 




Defense 

vs. Attack 

DD-HA 





Blue 


DD-DA 





Blue 


HD-HA 





Blue 


HD-DA 





Blue 





HA-HA 





Neither 





DA-HD 





Red 




Attack vs. 

HA-HD 

2.37 

2.50 

3.22 

3.48 

Red 


Defense 

DA-DD 





Red 


HA-DD 





Red 



- Ensure that the solution space is saturated. If the output data produced outcomes that heavily 
favors victory of one side over the other, then the meta-models generated from that analysis will be 
skewed in that direction. To ensure that this bias is removed, you must ensure the solution space 
has a nearly equally distribution of outcomes above and below the expected victory curve. A10% 
variation (40/60% split) is acceptable, any more and the DOE ranges above will need to be extended 
in the direction that favors the looser. 

- Each set of Mission attributes should result in a mean Pv of blue forces being within +/- 5% of the 
expected victory curves as described in the IRCPAT across the full range of force ratios ranges 
(Calibration low to Calibration high) as seen above, while attempting to maintain the relative 
hierarchy and order of each factor in relation to itself in other mission sets as seen in the table 
below. 

- Following the analysis and contour profiling exploration, record the Baselined Mission Attributes 
that minimize the variation between the Pv of the model and the reference tool across the 
calibration range of force ratios noted above. These will be used as the baseline factor settings in 
the model. 

- Execute a 1 x factor DOE of the updated model by varying only Force Ratio (ftBlueTanks), and verify 
that the updated model continues to have a good fit with the expected outcomes predicted by the 
reference tool across all potential Force Ratios. 


- For more accurate "Mission Attributes" calibration, repeat these steps for other mission sets. While 
this is may not be necessary, if the time is available, it is advised. 


The primary purpose of this analysis is to estimate a set of factor settings that will 
capture the impacts of the HA-HD mission set on the Pv, while maintaining the models 

calibration to the reference tool {+!- 5 %) across all possible FRs. To do this, I started by 
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ensuring that the solution space was saturated by plotting the responses (Py) for each DP 
across the range of potential FRs and comparing the results to the expected victory curve. 
Figure 60 shows the solution space plot for mission calibration. 



Figure 60. Mission Calibration DOE Solution Space Saturation 

As shown in Figure 60, the distribution of the DPs meets the IMDP criteria for 
being considered saturated, in that the total deviation between the upper and lower 
distributions is 10%. With saturation of the solution space validated, I could then use 
JMP to conduct the statistical analysis, regression, and contour profiling needed to 
determine the settings of the mission factors settings. The JMP contour profiler can be 
seen in Figure 61. 
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Horiz Vert Factor 
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RedCoocealment *- 

BluePh <= 
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o 
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o 


80 


70 


60 
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Figure 61. Contour Profiler of Mission Attributes 


By setting the maximum variance between mean Py and the expected Py from the 
IRCPAT to +/-5%, and then conducting TSE, I was able to identify specific factor 
settings for each of the eight mission factor settings that ensured consistent results across 
the FRs of 2.50 to 3.66. Thus, using these factor settings, the model was now calibrated 
to the reference tool, and should produce results that will fall within +/-5% of the results 
of the IRCPAT across all FRs of interest (2.50 to 3.66 in this example). These settings 
were then recorded for future inclusion in the model and can be seen in Table 32. 


Table 32. IMDP: A3b-4 (Results) 
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These factor settings capture the mission dependent advantages of the defending 
force, and were used in the development of the model. With the mission factor settings 
recorded, the next step was to execute a one factor DOE, where I varied the FR to verify 
that the output of the model mirrors the expected results of the IRCPAT across the full 
range of potential FRs. The results of this analysis can be seen in Figure 62. 



Figure 62. Mission Calibration Verification 


As shown in Figure 62, the verification DOE confirmed that the Mission factor 
settings identified previously produced nearly identical results between both the model 
and the IRCPAT when implemented. Thus, the model was calibrated with the reference 
tool and capable of producing results that fall within +/- 5% of the IRCPAT across all 
FRs. With mission calibration complete, I moved on to building more functionality into 
the model by accounting for system specific attributes and capabilities that were ignored 
in mission calibration. 

h. Part A3c: Agent Calibration 

The intent of this step is to link the model to the IRCPAT through the 
establishment of agent specific attributes that can delineate between the differences of the 
opposing tanks, capturing the advantage that the M1A2 will have over the T-80U while 
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maintaining the synchronization of the expected victory between the tool and the model. 
Table 33 outlines the steps of agent calibration. 


Table 33. IMDP:PartA3c 


1 

Update the reference tool by modifying it for the systems of interest, which will now differentiate 
between opposing forces. 

2 

Agent Calibration: Calibrate the Agent Attributes for each agent to account for variations in capabilities 
based on the Improved Force Analysis System Weights Spread sheets to create the final "Agent 
Attributes" to be used in the Model. 

3 

DOE: Execute a multi-factor DOE of the updated model by varying Force Ratio and any Agent attribute 
identified above in order to verify that the model continues to have a good fit with the expected 
outcomes predicted by the reference tool across all potential Force Ratios after agent calibration. 

4 

Analysis: Conduct analysis to establish the baseline agent factor settings. 


- Trade-space exploration to minimizing variation form expected outcomes. 

- Record the Calibrated Agent Attributes as factor settings for use in the model. 


I began agent calibration as before, by expanding the IRCPAT to account for the 
specific agents used for this demonstration, which included MlA2s for the Blue force and 
T-80Us for the Red force. Following the expansion, I modified the number for Blue tanks 
to maintain the expected Py of the Blue force at 50%, which can be seen in Figure 63. 
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Improved Relative Combat Power Assessment Tool (IRCPAT) 


Ratio ofFnendlyto Enemy 


Enemy Forces 




Armor (Ml A2) 




Tdnk(T-S0U) 


9 ^ 


184.00 


0.86 


17.20 


System Force Equivalent 


466.55 


S^slem Force Eguralent 


184.00 


External Force Equivalent 


107.50 


External Force Equivalent 


17.20 


Status of External C4I 


0.00 


Status of External C4I 


ESINQ Effects 


ESIllQ Effects 


Status of BR Deoradation 


External Systems 


External Systems 


100% SmaBal (ISR) 


Friendly Force Equivalent 


466.55 


Enemy Force Equivalent 


184.00 


Ratio of Friendly to Enemy 
2.536:1 


Ratio of Enemy to Friendly 
0 394 1 


Hasty Attack 

<- Mission -> 

Hasty Defense 

74 1 17.2% 

<- Estimated Losses -> 

35 3% 1 7 1 

0497 

<- Estimated % Victory (Curve 1) -> 

0503 

0510 

<- Estimated % Victory (Curve 2) -> 

0490 

0.526 

<- Estimated % Victory (Curve 3) -> 

0.474 

0.533 

<- Estimated % Victory (Curve 4) -> 

0 467 


Figure 63. IRCPAT (Agent Calibration) 


As shown in Figure 63, the IRCPAT was updated to account for the system level 
differences between the tanks of the opposing forces. Thus, a significant shift in the RCP 
was seen, which increased the expected Blue Py from 50% to roughly 80%, which was 
expected when considering the superiority of the M1A2 over the T-80U. To take 
advantage of this increase in RCP, I decreased the number of Blue tanks needed to 
maintain a Blue Py of 50% from 50 to just 43. Following the expansion of the IRCPAT, 
the IMDP directed me to execute agent attribute calibration, starting with system 
comparison of the opposing tanks; the steps of which can be seen in Table 34. 
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Table 34. IMDP: A3c-2 (System Comparison) 


2 Agent Calibration: Calibrate the Agent Attributes for each agent to account for variations in 
capabilities based on the Improved Force Analysis System Weights Spread sheets to create 
the final "Agent Attributes" to be used in the Model. 

- Compare the system attributes using the source documents of the reference tool, 
o Capture the differences between these agents in a table. These 5 areas, and 

associated factors, will be used to capture the differences between the systems, and 
include: 

1 System Attributes (speed) 

2 Protection Attributes (Enemy Ph, Concealment) 

3 Weapons Attributes (Ph) 

4 Secondary Weapons Attributes (Ph) 

5 C4SI (Td, Pc) 

o The completed assessment of the system differences can be seen below. In this 
example, we compare an MIA2 to a T80U. 


Attribute 

Weight 

M1A2 

T-80U 

+/-5% 

Modeled? 

Calibrate? 

System 

0.25 

0.17 

0.17 

0.0% 

Yes 

No 

Protection 

0.35 

0.38 

0.28 

26.3% 

No 

Yes 

Weapons 

0.35 

0.21 

0.22 

-4.8% 

Yes 

No 

Aux Weap 

0.05 

0.08 

0.05 

37.5% 

No 

No 

Inter C4I 

N/A 

0.25 

0.201 

19.6% 

No 

Yes 

F.E.xlO 

1.00 

10.90 

9.21 





o Note: Internal C4I comes from Step A2b (Internal C4I FE). and accounts for roughly 
23% of the systems combat power. External C4I will be addressed in the next section 
(External Dependencies Calibration). 

o Determine if the model needs to be calibrated to account for them. Ifthe system 
attributes are equivalent (+/'5%), or are explicitly modeled, then no calibration is 
needed. If calibration is needed, identify the key combat factors that can be used to 
calibrate each agent in these areas, less is better. Avoid factors that are 
known/fixed.-.like firing rate, number rounds, ect. Only calibrate factors that are not 
accounted for by the model or quantified by real world data...i.e. only factors that you 
are using as a baseline for all agents. 

o Quantify the changes to agent attributes needed to represent the system differences. 
M1A2 are 26.3% more protected than the T-80U, so reduce the T-80s Ph by 26.3% 
(Weighted by 70% to account for the weapon weight (35%) and the protection weight 
(35%)) (Main Gun). Because "protection" is not accounted for by MANA in any specific 
way, this calibration is valid. 

M1A2 Aux Weapons are 37.5% better. If weapons are modeled individually, 
disregard, validation is not needed. If using a base system for both agents, like a 50 
Cal, then calibration is valid, and user should increase the RoF of the M1A2 50 Cal by 
37.5%. Because 50 calls are irrelevant in this example, this calibration is not needed. 
M1A2 has 19.6% better sensors. If sensor ranges/Td.Pc are known, disregard, 
validation is not needed. If using a common base sensor for both agents as in this 
example, then calibration is valid, and user should increase the Td,Pc of the T-80U by 
19.6%. Weight this for the TA-80 status as a Tier 2 system (40% contribution of C4I, 
70% of which is internal). Because of the small range of Td. add in a few seconds. 


The first half of agent calibration supported the comparison of opposing agents by 
identifying the difference between the tanks in terms of modeling factors used in the 
model. Following my comparison of the tank attributes of both the M1A2 and the T-80U 
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in the FA-SWN, I made a determination on which agent factors I needed to calibrate. 
Because the system and weapons attributes of both tanks were within +/- 5% of each 
other, calibration of these factors was not needed. Because auxiliary weapons were not 
used in this demonstration, that difference did not need to be accounted for either. This 
left just two factors that were both significant and not being modeled: protection and 
internal C4I. Thus, the IMDP recommended the calibration of Red Ph(-), Td(+), and Pc(-) 
to account for these system level differences. Following system comparison, the next step 
was to quantify the delta that would need to be applied to each factor; the steps of which 
can be seen in Table 35. 


Table 35. IMDP: A3c-2 (Quantification) 


- Modify the model to account for these system differences. 

o Instantiate the table below using the assessments from above. Note, this is a rough 
assessment, meant to give us a starting point for establishing the bounds of the DOE, 
it doesn't need to be perfect. So if unsure, error towards a larger design space, the 
following steps will support the scoping of the range, 
o Note: When selecting a choosing individual factors it is sometimes possible for 
anomalies in later steps of the calibration to occur. These anomalies, while rare, 
typically happen when factors are calibrated separately. To avoid such anomalies, 
the IMDP recommends always calibrating factors in pairs. For example, the previous 
steps say to reduce Red Ph by 26.3%. The IMDP suggest you do this by splitting the 
delta, reducing Red Ph by 13.1% and Increasing Blue Ph by 13.1%, then calibrate both 
factors. This should be done for all factors. For simplicity of this example, I forgo 
this recommendation because I already know that no anomalies will result. 


Factor 

Mission 

Attribute 

Cal 

Modifier 

Weight 

Agent 

Attribute 

Note 


Blue Agent Speed 

30 



30 


From Step 

A3b 

(Mission) 

Red Agent Speed 

0 

- 

- 

0 


Blue RoF 

12 

- 

- 

0 


Red RoF 

26 

- 

- 

0 


Blue Concealment 

10 

- 

- 

10 


Red Concealment 

65 

- 

- 

65 


Blue Ph (at 3500m) 

0.150 

- 

- 

0.150 


Red Ph (at 3500m) 

0.400 

0.263 

0.70 

0.326 

Main Gun 

Blue Td (4000m) 

8.000 

- 

- 

8.000 


New 

(Agent) 

Red Td (4000m) 

8.000 

0.196 

0.70 

9.098 

Veh 

Blue Pc (per det) 

0.350 

- 

- 

0.350 


Red Pc (per det) 

0.350 

0.196 

0.80 

0.295 

Veh 


o Of these factors. Speed, RoF, Cone, and Ph have already been mission calibrated in the 
model, thus there modification is simple. Because Td and Pc have not been calibrated 
yet, careful attention will need to be paid to their implementation and selection of 
settings. 
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Calibration modifiers for each identified agent factor were transferred into the 
tool, along with the weight for that source of combat power based on the analysis of the 
previous step. The tool then recommended a modified bounds for the DOE for each of 
these agent attributes. Following the quantification of the factor modification, I was able 
to finalize agent calibration following the steps in Table 36. 


Table 36. IMDP: A3c-2 (Bounding) 


- Transfer these factor settings and establish appropriate DOE ranges in the table below, 
o The FR ranges will need to be adjusted to maintain the 40% to 95% victory calibration 
bounding. To do this, use the IRCPAT instantiated at the begging of this step to adjust 
the number of agents in order to identify the number needed to achieve 40%, 50%, 

90%, and 95% victory. For this example, this was 40,43,55,and 59 tanks, 
o "Best” ranges (High or low), should be fixed based on the mission attributes above. 

I.e., the DOE should not allow values that would increase the capabilities of attributes 
which we are trying to reduce. The other side (high or low) should use the new agent 
attribute from above. Because we are not accounting for weighting, we are likely 
overestimating the impact of these attributes, but the DOE should account for that if 
property bounded. 



DPs 

Factors 

Low (40%) 

Model 

High (95%) 

FR 

Attribute 

1 

4 Blue Tanks 

40 

43 

59 

Mission 

Attribute 

2 

Red Ph (at 3500m) 

0.326 

0.350 

0.400 

Attribute 

3 

RedTd 

8 

9 

11 

4 

Red Pc 

0.295 

0.325 

0.350 


As shown in Table 36, I started by adjusting the FR attributes. Because the 
relative combat power of the Blue tanks increased, the number of tanks had to be reduced 
to maintain the 40% to 95% Py trade-space of interest to me. Thus, the range of Blue 
tanks decreased from 46-83, as used in the mission calibration, to just 40-59. Next, the 
mission and agent attribute DOE ranges were altered based on the desired impact of the 
modification. For Ph and Pc, this required a decrease in capability, and thus, I modified 
the lower bound of the DOE range while fixing the upper bound. For Td, the opposite was 
true, and an increase was needed, so I fixed the lower bound and extended the upper 
bound. With the Agent attributes identified and the DOE ranges established, I executed 
the DOE following the steps outlined in Table 37. 
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Table 37. IMDP: A3c-3 


3 DOE; Execute a multi-factor DOE of the updated model by varying Force Ratio and any Mission/Agent 
attributes identified above in order to verify that the model continues to have a good fit with the 
expected outcomes predicted by the reference tool across all potential Force Ratios after agent 
calibration. 

- Set your Design Points based on the work from above, modifying just the Mission/Agent attributes 
noted above. Ensure you provide adequate trade space to explore each factor setting. 


0 

stack 1 ( vs. Defense) 




DPs Factors Low 

High 



FR 

Attributes 

1 

# Blue Tanks 

40 

59 



Mission 

Attributes 

2 

Red Ph (at 3500m) 

0.326 

0.400 



Agent 

Attributes 

3 

Red Td 

8 

11 


4 

Red Pc 

0.295 

0.350 


Select and Build your Design. 



0 

Determine the number of Factors (4 in this example). 




0 If the number of factors is 13 or less, use a 2nd Order NOLH design. If it is greater than 13 up to 
29, use a NOLH design. Both the NOLHdesigns.xIsx and the MacCalman-2ndOrderNOLH design 
tools can be found at https: //harvest.nps.edu/. 

0 Download the DOE tool and input the values from the previous step. 

0 Consider conducting a screening design to determine significance, this may allow you to reduce 
the number of factors, and thus, use a more detailed 2nd order design in subsequent analysis. 

- Stack the design . 

0 Stack the design to achieve better resolution within the design space as well as the solution 
space, which will provide better fitting models. Remember to vary the columns when stacking. 

- Execute the DOE. 

0 For my work I provided the following files to the NPS SEED Center in order to run my model on 
the advanced computing cluster. 



1 

MANA Model .xml 


2 

The Design developed using the DOE above (translation to the .csv is needed) 


3 

The Study .xml script (this executes the model and design on the cluster) 


- The SEED center can assist in preparing these documents for execution if needed. 


As with mission calibration, I chose to use a simpler NOLH design for this 
demonstration rather than the order NOLH design the IMDP suggests. To increase the 
design saturation, I used a nine-factor design rather than a four-factor design, and after 
stacking nine ties and replicating 100 times, this 33 DP design resulted in a 29,700 DP 
simulation run. Once the output data was returned, the steps outlined in Table 38 were 
used to conduct analysis. 
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Table 38. IMDP: Part A3c-4 


4 






Tanks 

FR before 

FR now 




#BlueTanks* 10.85 


59 

2.95 

3.48 




#RedTanks * 9.20 


20 



Conduct analysis of the DOE output in order to establish the Agent Attributes for the HA-HD 
mission which results in the least variation between the model and the IRCPAT expected Blue 
Victory % across the full range of potential force ratios. 

- For this work, I used JMP to conduct regression analysis, and then used the contour 
profilers to identify Agent settings for each mission set that kept variation of the mean 
less than +/-5% from the expected victory based on victory curve #4. 
o Now that agent differences are now being accounted for, we can no longer use #Blue 
Tanks/#RedTanks to represent the Force Ratio when creating a FR column in JMP. 
Thus, we need to us a formula to calculate the actual FR as follows: 


o This difference between FRs captures the system level advantages of the systems being 
modeled. Otherwise, the analysis is the same as in the last step. These numbers are 
based on the FE from the reference tool. 

Ranges for analysis should be based on the mission set. Below is the suggested Force 
Ratio Ranges for each mission set, both bounding and calibration, based on victory curve 
4 of the IRCPAT. Again, these will need to be adjusted to account for the new Force Ratios 
based on Agent attributes. The FRs themselves should remain constant, only the # tanks 
has changed, to reflect the increased capability of tanks...thus, a smaller number of Blue 



Force Ratios for % Victory 



Mission 

Set 

Bound 
Low (40%) 

Calibrate 
Low (50%) 

Calibrate 

High 

Bound 

High 



HA-HD 

2.37 

2.50 

3.22 

3.48 




- Ensure that the solution space is saturated. If the output data produced outcomes that 
heavily favors victory of one side over the other, then the meta-models generated from 
that analysis will be skewed in that direction. To ensure that this bias is removed, you 
must ensure the solution space has a nearly equally distribution of outcomes above and 
below the expected victory curve. A 10% variation (40/60% split) is acceptable, any more 
and the DOE ranges above will need to be extended in the direction that favors the looser 

- Each set of Agent attributes should result in a mean victory of blue forces being within +/- 
5% of the exijected victory curves as described in the IRCPAT across the full range of force 
ratios ranges (Calibration low to Calibration high) as seen above, while attempting to 
maintain the relative hierarchy and order of each factor in relation to itself in other 
mission sets as seen in the table below. Thus, try to limit the number of factors adjusted. 
In this example, I choose to modify just Red Ph to finalize the calibration, increasing it 
from 0.257 to 0.345. This is still below the original Mission attribute of 0.4, so this 
modification just limits the size of the degradation due to agent calibration. Because Ph 
was the most significant factor, it is easy to see how I simply overestimated the impact of 

- Following the analysis and contour profiling exploration, record the Baselined Agent 
Attributes that minimize the variation between the expected victory of the model and the 
reference tool across the calibration range of force ratios noted above. These will be 
used as the baseline factor settings in the model. 

- Execute a 1 x factor DOE of the updated model by varying only Force Ratio (#Blue Tanks), 
from 40 to 59 in this example, and verify that the model continues to have a good fit with 
the expected outcomes predicted by the reference tool across all potential Force Ratios. 

- For more accurate "Agent Attributes" calibration, repeat these steps for other agent types 
if other agents account for greater than 25% of the total RCP. If not, you can disregard at 
this point, all other agents will be implicitly calibrated in the operational model during 
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The purpose of agent calibration analysis is to estimate a set of factor settings that 
will capture the impacts of the system level differences between the tanks within the 
model, while maintaining calibration of the model output to the IRCPAT (+/- 5%) across 
all possible FRs. The first thing I did here was to change how the FR was calculated in 
my analysis. Previously, when the systems were identical, calculating the FR was simply 
done by dividing the number of Blue tanks by the number of Red tanks. Now that the 
systems were no longer equivalent, I modified the calculation by multiplying the number 
of tanks by their respective FE. Failure to do so would have resulted in flawed analysis. 
Next, I ensured that the solution space was sufficiently saturated. Figure 61 shows the 
solution space plot for agent calibration. 



Figure 64. Agent Calibration DOE Solution Space Saturation 

As shown in Eigure 64, the distribution of the DPs meets the IMDP criteria for 
being considered saturated, in that the total deviation between the upper and lower 
distributions about the expected curve is 10% or less. Initially, I did not achieve adequate 
saturation, and I was forced to iterate my DOE ranges, specifically by expanding the 
lower bound of Red Ph and the upper bound of Red Td, until I achieved the saturation 
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seen in Figure 67. With saturation of the solution space validated, I then used the IMP 
contour profiler seen in Figure 65 to determine the Agent factors settings. 
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03' 
2.85' 
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25 

40% 50% 


2.75 3 
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Figure 65. Contour Profiler of Agent Attributes 


With these attribute settings, I was able to achieve less than 5% deviation in the Pv 
between the model and the IRCPAT across all FRs of interest, and less than 2% deviation 
across most FRs. These attributes capture the system dependent advantages gained from 
superior systems, and would be used in the future during the development of the final 
model. Following the calibration, I recorded the agent attributes in Table 39. 
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Table 39. IMDP: A3c-4 (Results) 


HA 


Cone 

10 

65 

Ph 

0.15 

0.345 

RoF 

12 

26 

Speed 

30 

0 

Pc 

0.350 

0.300 

Td 

8 

10 


With the agent factor settings recorded, my next step was to verify the results by 
executing a one factor DOE, where I varied the Force Ratio (#Blue Tanks) to verify that 
the output of the model mirrors the expected results of the IRCPAT across the full range 
of potential FRs. The results of my analysis can be seen in Figure 66. 



Figure 66. Agent Calibration Verification 


My verification DOE confirmed that the agent factor settings identified previously 
produced nearly identical results in both the model and the IRCPAT. Thus, the model 
was now calibrated with the reference tool, and capable of producing results that fall 
within +/- 5% of the IRCPAT across all FRs of interest. With agent calibration 
completed, I expanded the model to account for the external dependencies of the system. 
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i. Part A3d: External Dependency Calibration 

The intent of this step was to link the model to the IRCPAT through the 
establishment of attributes that would represent the system level dependencies on 
external sources for the generation of internal combat power. By calibrating external 
dependencies I was able to capture the contributions of superior external support sources 
and structures of opposing forces while maintaining the synchronization of the Py 
between the tool and the model. Table 40 outlines the steps of ExtDep calibration. 


Table 40. IMDP: Part A3d 


1 

Update the reference tool by setting the external dependencies of each force to 100%, which will now 
account for the external dependencies of the system for generating internal combat power. 

2 

External Dependency Calibration: Calibrate the ExtDep Attributes for each agent to account for 
variations in system dependencies on external sources of internal combat power to create the final 
"ExtDep Attributes" to be used in the Model. 

3 

DOE: Execute a multi-factor DOE of the updated model by varying Force Ratio and any ExtDep attribute 
identified above in order to verify that the model continues to have a good fit with the expected 
outcomes predicted by the reference tool across all potential Force Ratios after ExtDep calibration. 

4 

Analysis: Conduct analysis to establish the baseline ExtDep factor settings. 

- Trade-space exploration to minimizing variation from expected outcomes. 

- Record the Calibrated ExtDep Attributes as factor settings for use in the model. 


I began ExtDep calibration by expanding the IRCPAT to account for the external 
dependencies of the M1A2 and the T-80U. Specifically, I set the status of external C4I to 
100% for both forces. Then, I updated the number of Blue tanks to maintain the Py of the 
Blue force at 50%. My expanded IRCPAT can be seen in Eigure 67. 
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Improved Relative Combat Power Assessment Tool (IRCPAT) 
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Figure 67. IRCPAT (ExtDep Calibration) 


As shown in Figure 67, the IRCPAT was updated to account for the system level 
dependencies on external support for the generation of internal combat power. Thus, 
because of the greater contribution from external sources for the M1A2 compared to the 
T-80U, a significant shift in the RCP was seen, which increased the expected Blue Py 
from 53% to roughly 75%. I took advantage of this increased RCP by decreasing the 
number of Blue tanks needed to maintain the Blue Py at 50%, which resulted in a decrease 
from 43 to just 38 tanks. Following the expansion of the IRCPAT, the IMDP directed me 
to execute ExtDep attribute calibration, starting with the comparison of the external 
dependencies of the opposing systems, the steps of which are described in Table 41. 
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Table 41. IMDP: A3d-2 (ExtDep Comparison) 


2 External Dependency Calibration: Calibrate the final ExtDep attributes to identify the steady state 
settings for these factors (to include the contributions from external dependencies on internal metrics of 
combat power) that cause no disruption to the outcomes of the model and reference tool. These will 
act as the reference from which future manipulation of these factors will be made. 

- At this point, the model (w/o ext dependencies) matches the IRCPAT (w/o ext dependencies), thus, 
we have baselined the IRCPAT and model for system level combat power. Now we must account for 
the systems internal dependencies on external contributions to combat power. This is already done 
in the IRCPAT, but now we must calibrate the model to the IRCPAT, this time for external 


dependencies. 

- Copy the calibration attributes ranges from step A3c here. 




Base Line (0% Ext Cent) 


Expanded (100% Ext Cent) 



HA 


HA 



Cone 

10 

65 


• 




Ph 

0.15 

0.345 





RoF 

12 

26 





Speed 

30 

0 





Pc 

0.350 

0.300 





Td 

8 

10 









o These base line attribute values will represent the worse case scenario (0% contribution to 
internal combat power in the model from external sources) The goal of this step is to figure out 
the best case scenario (100% contribution). 


- Insert the FE and Ext dependencies for the agent being calibrated from part A2b. 





FE 

ext 

tot 

% 


M1A2 

10.85 

2.5 

13.35 

0.187 


TA-80U 

9.2 

0.86 

10.06 

0.085 




0 The increase to combat power of both forces will need to be accounted for in the model to 
represent the contribution from external sources. Thus, we will need to modify the baseline 
calibration attributes to account for this increase in combat power, and then use these new 
values as the 1(X)% external contributions to internal combat power. The difference between 
these two sets of values will drive the meta-models used to represent the impacts from internal 
dependencies on external sources of combat power in the model. 


This first part of ExtDep calibration supported the comparison of opposing agents 
by identifying the dependencies of the tanks on external sources for the generation of 
combat power. By determining the percentage of total combat power for each tank 
derived external to the system boundary, I was able to estimate a general modification 
factor that could be used on specific agent attributes in the model to increase the overall 
effectiveness of both tanks due to the contributions from external sources. Eollowing 
ExtDep comparison, the next step focused on identifying the mission and agent factors 
that would be modified to represent this increase in system level combat power, the steps 
of which can be seen in Table 42. 
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Table 42. IMDP: A3d-2 (ExtDep Attribute Selection) 



Identify the attributes from above that can be used to represent the contributions from eternal 
systems on internal combat power. Because external dependencies revolve mostly around C41,1 
would suggest attributes that would likely be impacted by it. 



C4I Attributes: Cone Ph BoP Speed Pc Td 


0 

Because one force is likely stationary, speed is not very useful. 


0 

Because Td is currently set at 8/10 sec respectively, it is limited on the low end, and thus does 
not have much room for improvement, and thus, is likely just a secondary factor at best. 


0 

Pc has strong ties to C4I, and a lot of wiggle room, thus, is a good primary factor for calibration. 


0 

While Ph is not directly impacted by C4I, it is indirectly, and thus a usable modifier if the other 
factors don't have enough power to make the required changes, to be safe, include it in the 

DOE for now, but try to minimize changes. 


As shown in Table 42, I conducted a logical inspection of the potential factors 
that could be modified to represent the desired impacts from the application of external 
dependencies for the tanks. Following my assessment, I determined that Ph, Td, and Pc 
were all potential candidates, but decided that Td and Pc were more appropriate with 
regard to C4I, and should have more than enough flexibility in their ranges to capture the 
impacts which I was interested in representing in the model. Thus, I transferred these two 
factors along with the corresponding attribute values into the tool provided by the IMDP 
described in Table 43. 
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Table 43. IMDP: A3d-2 (Bounding) 


- Transfer these factors and the Agent attribute values for each in to the format below. The tool will 
produce a DOE range for the calibration of external contributions to internal combat power. Note, 
depending on the direction of improvement, the high or low of each attribute will remained fixed at 
the baseline values, this will ensure that no improvement will cause a reduction in capabilities. 




Factor 

Agent 

Attribute 

Cal 

Modifier 

ExtDep 

Attribute 

DOE Low 

Model 

DOE High 

Blue Ph (3500m) 

0.150 

0.187 

0.178 

0.150 

0.178 

0.206 

Red Ph (3500m) 

0.345 

0.085 

0.374 

0.345 

0.374 

0.404 

Blue Td (4000m) 

8 

0.187 

6.502 

4 

6 

8 

Red Td (4000m) 

10 

0.085 

9.145 

8 

9 

10 



Blue Pc (per det) 

0.350 

0.187 

0.416 

0.350 

0.416 

0.481 

Red Pc (per det) 

0.300 

0.085 

0.326 

0.300 

0.326 

0.351 



0 

Note: As we saw in Agent calibration, the IMDP recommends that all factors be calibrated in 
pairs, as shown here. Additionally, ExtDep calibration must take place on factors that have 
already been calibrated in either mission or agent calibration, attempting to calibrate factors in 
this step that have not been calibrated previously risk introducing anomalies in the resulting 
meta-models. Thus, for this example, 1 am limited to the use of just Red Ph, Td, and Pc. 

Because 1 believe that the use of Ph is unnecessary to capture the impacts that 1 am looking to 
represent, 1 will not be using it. Thus, the demonstration table looks like this. 




Factor 

Agent 

Attribute 

Cal 

Modifier 

ExtDep 

Attribute 

DOE Low 

Model 

DOE High 

Red Td (40(X)m) 

10 

-0.102 

11.020 

10 

11 

14 



Red Pc (per det) 

0.300 

-0.102 

0.269 

0.208 

0.269 

0.300 



0 

Because 1 am just modify the red attributes from Agent calibration (Red Td and Pc) in this 
example (which is purely for demonstration purposes), 1 will need to take the difference 
between the ExtDep of both systems, and apply it to the Red force. In this case, an overall 
increase of 0.102 to blue combat power is expected. Thus, by applying this as a negative 
modifier to red Td and Pc, we can achieve the desired differential. While this will not allow us to 
independently modify Blue or Red dependencies on ExtDep, for this demonstration it is 
sufficient, and will demonstrate the dependence of both forces on external sources of combat 
power as long as that dependency scales equally. 


0 

As before, the FR ranges will need to be adjusted to maintain the 40% to 95% victory calibration 
bounding. To do this, use the IRCPAT Instantiated at the begging of this step to adjust the 
number of agents (Blue Tanks) in order to identify the number needed to achieve 40%, 50%, 

90%, and 95% victory. For this example, this was 36, 38, 49, and 53 tanks. 


While the IMDP recommended the use of six factors to represent external 
dependencies of the opposing forces, the approach I took with this demonstration allowed 
me to use just two. To identify the ExtDep attribute DOE ranges needed to achieve the 
desired relative improvements to Blue Pv due to external dependencies, Td and Pc of the 
Red force would need to be decreased. To do this I fixed of one side of the DOE range 
for each factor using the current attribute settings, and used the tool provided by the 
IMDP to establish the other bound. Next, to account for the increase in Blue combat 
power due to the superior application of external sources of combat power, I reduced the 
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number of Blue tanks to maintain the 40% to 95% Py trade-space of interest to me. Thus, 
I was able to reduce the number of Blue tanks needed in the DOE from 40-59 as used in 
the agent calibration, to just 36-53. With the ExtDep attributes identified, ER determined, 
and the DOE ranges established, I executed the DOE as outlined in Table 44. 


Table 44. IMDP: A3d-3 


3 DOE: Execute a multi-factor DOE of the updated calibration model by varying Force Ratio and the 
ExtDep attributes identified above in order to verify that the updated model continues to have a good 
fit with the expected outcomes predicted by the reference tool across all potential Force Ratios. 

- Set your Design Points based on the work from above, modifying just the ExtDep attributes noted 
above. Ensure you provide adequate trade space to explore each factor setting. 



0 

stack 1( vs. Defense) 





DPs Factors Low High 





FR 

Attributes 

1 

#Blue Tanks 

36 

53 





ExtDep 

Attributes 

2 

Blue Ph (3500m) 

0.15 

0.206 



3 

Red Ph (3500m) 

0.345 

0.404 



4 

Blue Td (4000m) 

4 

8 





5 

Red Td (4000m) 

8 

10 



6 

Blue Pc (per det) 

0.350 

0.481 





7 

Red Pc (per det) 

0.300 

0.351 




0 

For this demonstration, the following simplified design was used. 





FR 

Attributes 

1 

#Blue Tanks 

36 

53 





ExtDep 

Attributes 

2 

Red Td (4000m) 

10 

14 





3 

Red Pc (per det) 

0.208 

0.300 










- 

Select and Build your Design 






0 

Determine the number of Factors (3 in this example) 





o If the number of factors is 13 or less, use a 2nd Order NOLH design. If it is greater than 13 up to 
29, use a NOLH design. Both the NOLHdesigns.xIsx and the MacCalman-2ndOrderNOLH design 
tools can be found at https: //harvest.nps.edu/ 

0 Download the DOE tool and input the values from the previous step 

o Consider conducting a screening design to determine significance, this may allow you to reduce 
the number of factors, and thus, use a more detailed 2nd order design in subsequent analysis. 

- Stack the design 

0 Stack the design to achieve better resolution within the design space as well as the solution 
space, which will provide better fitting models. Remember to vary the columns when stacking. 

- Execute the DOE 

0 For my work I provided the following files to the NPS SEED Center in order to run my model on 
the advanced computing cluster. 

1 MANA Calibration Model .xml 

2 The Design developed using the DOE above (translation to the .csv is needed) 

3 The Study .xml script (this executes the model and design on the cluster) 

- The SEED center can assist in preparing these documents for execution if needed. _ 


As with agent calibration, I chose to use a simpler NOEH design for this 
demonstration rather than the 2"‘* order NOEH design the IMDP suggests. To increase 
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saturation I used a nine-factor design, and after stacking nine times and replicating 100 
times, the 33 DP design resulted in a 29,700 DP simulation run. Once the output data was 
returned, I used the steps outlined in Table 45 to conduct analysis. 


Table 45. IMDP: Part A3d-4 


4 Conduct analysis of the DOE output in order to establish the modifications to the calibration attributes 
needed to represent the impacts from internal dependencies on external contributors to combat power, 
which results in the least variation between the model and the IRCPAT expected Blue Viaory % across the 
full range of potential force ratios. 

- For this work. I used JMP to condun regression analysis, and then used the contour profilers to identify 
modified calibration attributes settings for each ExtOep factor that kept variation of the mean less than 
*/-5% from the expected viaory based on the IRCPAT viaory curve #4. 

o Now that ExtOep are now being accounted for, we can no longer use the previous FR formula to 
represent the Force Ratio when creating a FR column in JMP. Thus, we need to us a formula to 


calculate the aaual FR as follows: 




Tanks 

FR before 

FR now 



#Blue Tanks * (10.85 - 2.50) 

53 

3.13 

3.52 



#RedTanks * (9.20 ♦ 0.86) 

20 



o This difference between FRs captures the system level advantages of the systems being modeled. 
Othenwise, the analysis is the same as in the last step. These numbers are based on the FE from 
the reference tool. 

Ranges for analysis should be based on the mission set. Below is the suggested Force Ratio Ranges for 
each mission set, both bounding and calibration, based on viaory curve 4 of the IRCPAT. Again, these 
will need to be adjusted to account for the new Force Ratios based on ExtOep attributes. The FRs 
themselves should remain constant, only the u tanks has changed, to reflea the increased capability of 
tanks...thus, a smaller number of Blue Tanks can achieve the same FR as previously. 




Force Ratios for % Viaory 



Mission Sec 

Bound Low 
(40%) 

Calibrate 
Low (50%) 

Calibrate 
High (90%) 

Bound High 
(95%) 



HA-HD 

2.37 

2.50 

3.22 

3.48 


• Ensure that the solution space is saturated. If the output data produced outcomes that heavily favors 
viaory of one side over the other, then the meta-models generated from that analysis will be skewed in 
that direaion. To ensure that this bias is removed, you must ensure the solution space has a nearly 
equally distribution of outcomes above and below the expeaed viaory curve. A10% variation (40/60% 
split) is acceptable, any more and the DOE ranges above will need to be extended in the direaion that 
favors the looser (without exceeding original Max calibration Attributes). This is an iterative process. 

- Each set of calibration attributes should result in a mean viaory of blue forces being within */■ 5% of 
the expeaed viaory curves as described in the IRCPAT across the full range of force ratios ranges 
(Calibration low to Calibration high) as seen above. Ensure that the analysis includes the external 
dependencies that have up until this step been ignored in the IRCPAT and analysis. This will require an 
updated Force Ratio formula as well as the updated IRCPAT. 

- Following the analysis and contour profiling exploration, record the Expanded calibration Attributes 
that will be used to account for the external dependencies of the system that minimize the variation 
between the expeaed viaory of the model and the reference tool across the calibration range of force 

- Execute a 1 x faaor DOE of the updated calibration model (ExtDep expansion) by varying only Force Ratio 
(PBIueTanks), and verify that the updated model continues to have a good fit with the expeaed 
outcomes prediaed by the reference cool across all potential Force Ratios. This will represent your 
100% External dependencies data source for meta-model development. 

- For more accurate "ExtDep Attributes" calibration, repeat these steps for other mission sets. While this 


The purpose of ExtDep calibration analysis was to estimate a set of factor settings 
that can represent the impacts of the system level dependencies on external sources for 
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the generation of internal eombat power, while maintaining the models calibration to the 
IRCPAT (+/- 5%) across all FRs. The first thing I needed to do here was to change how 
FR was calculated in my analysis. I did this by modifying the formula for calculating FR 
by including external dependencies into the tank FE prior to its multiplication with the 
number of tanks. Next, I ensured that the solution space was sufficiently saturated, the 
plot for which can be seen in Figure 68. 



Figure 68. ExtDep Calibration DOE Solution Space Saturation 

As shown in Eigure 68, the distribution of the DPs did not quite meet the IMDP 
criteria for being considered saturated because the total deviation between the upper and 
lower distributions about the expected curve was 12%, 2% greater than required. 
Regardless, I felt that it was close enough for this demonstration for use, and as long as 
the DOE and the verification supported that claim it would be acceptable. Eollowing my 
tentative assessment of a saturated solution space, I determined the settings of the ExtDep 
factors settings by using IMP, the contour profiler for which can be seen in Eigure 69. 
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Figure 69. Contour Profiler of ExtDep Attributes 


Using the two ExtDep attribute settings shown here, I was able to achieve a less 
than 3% deviation between the Py of the model and the IRCPAT across all ERs of 
interest. Thus, using these factor settings, the model should now be calibrated to the 
reference tool, and produce results that will fall within + 1 - 5 % of the results of the 
IRCPAT. These settings were then recorded in Table 46. 


Table 46. IMDP: A3dc-4 (Results) 



These factor settings capture the relative dependency of the opposing forces on 
eternal sources for the generation of internal combat power, and will be used in the future 
during the development of the final model. Because I chose to use a simplified 
demonstration of the IMDP, I was only able to capture the relative impacts of external 
contributions to internal combat power on the model. Thus, while I was able to modify 
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the contribution from 0 to 100% for both forces while accurately representing the impacts 
that external dependencies had on the forces, I could not do so independently. To do so, I 
should have executed the IMDP as indented, not as I did for this simplified demonstration 
which was intended to minimize resources and complexity. While this was not optimal, 
because my decision saved significant time and effort, I believed it was more than 
justified. With the ExtDep factor settings recorded, the next step was to verify the results 
by executing a one factor DOE, where I varied the number of Blue tanks to verify that the 
Pv of the model mirrors that of the IRCPAT across the full range of potential ERs. The 
results of my analysis can be seen in Eigure 70. 



Eigure 70. ExtDep Calibration Verification 


As shown in Eigure 70, my verification DOE confirmed that the ExtDep factor 
settings identified previously produced nearly identical results between the model and the 
IRCPAT. Thus, the model was now ExtDep calibrated with the reference tool, and 
capable of producing similar results (+/- 5%) as the IRCPAT across all ERs of interest. 
With ExtDep calibration complete, I was able to improve the model further by accounting 
for the contributions from the ESINQ factors of interest. 
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Part A3e: ESINQ Calibration 


The intent of this step was to calibrate any ESINQ factors of interest that were not 
previously calibration during mission, agent, or external dependency calibration. Unlike 
previous calibration steps, the purpose here was not to establish calibration values, 
because ESINQ values were established during baseline activities. During this step, the 
focus was on validating the ESINQ values that had already been used in the calibration 
process. Table 47 outlines the steps I used for ESINQ calibration. 


Table 47. IMDP:PartA3e 


1 

ESINQ Calibration: Calibrate the ESINQ Attributes to identify the steady state settings for these factors 
that cause no disruption to the outcomes of the model and reference tool. These will act as the 
reference from which future manipulation of these factors to implement ENSIQ effects will be made. 

2 

DOE: Execute a multi-factor DOE of the updated model by varying Force Ratio and any ESINQ attribute 
identified above in order to verify that the model continues to have a good fit with the expected 
outcomes predicted by the reference tool across all potential Force Ratios after agent calibration. 

3 

Analysis: Conduct analysis to establish the baseline ESINQ factor settings. 

- Trade-space exploration to minimizing variation from expected outcomes. 

- Record the Calibrated ESINQ Attributes as factor settings for use in the model. 


Unlike previous calibration steps, there was no need to start this step by updating 
the IRCPAT. The IRCPAT cannot address ESINQ factors, though at the conclusion of 
this demonstration I will have the meta-models necessary to do so. This application of the 
IMDP is outside of the scope of this demonstration, but will be discussed in Chapter V: 
Application. Thus, I began ESINQ calibration by determining which ESINQ factors still 
required calibration, the steps of which were found in Table 48. 
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Table 48. IMDP: A3e-1 (ESINQ Calibration Assessment) 


1 ESINQ Calibration: Calibrate the ESINQ Attributes to identify the steady state settings for these factors 
that cause no disruption to the outcomes of the model and reference tool. These will act as the 
reference from which future manipulation of these factors to implement ENSIQ effects will be made in 
Part B. 

- Copy the ESINQ Attributes identified from the EEMM Step Ale here. “For demonstration reasons, 
I added in a few extra ESINQ effects to fully explain the process. 



Input Factor 

Mission 

Agent 

ExtDep 

Calibrate? 

Dependent 

Ave Det Time (sec) 


Yes 

Yes 

No 

“YeT 

Prob Classification (%) 


Yes 

Yes 

No 

Yes 

«• 

Latency (sec) 




■■ 

No 

*» 

Reliability (%) 





No 

«« 

Ph per Discharge (%) 

Yes 

Yes 


No 

Yes 


0 Disregard any factors that have already been captured in mission, agent, or ExtOep calibration. 
In this example, this includes: 

Mission Calibration: Ph 

Agent Calibration: Ph, Td, Pc 

ExtOep Calibration: Td, Pc 

0 Assess the system dependency on the ESINQ effect. For direct fire systems, with shooting based 
on Agent/squad SA, they can see the target and have limited dependencies on latency and 
reliability. For indirect systems, which shoot based on inorganic SA, are not as much effected by 
Td or Speed, but are heavily dependent on Latency and reliability and its impact to Ph. 

0 If all ESINQ factors are accounted for in either Mission, Agent, or ExtOep calibration, or not 
necessary due to lack of dependency, this step is not necessary, all ESINQ factors have already 
been calibrated for steady state, and the user can move on to the next step. 

0 In this example, an M1A2 is a direct fire system, and thus, not sufficiently dependent on Latency 
or reliability, and thus, these factors would not need to be calibrated. To keep this 
demonstration moving forward, lets pretend we are looking at an indirect fire system instead. 
Thus, calibration of Latency and reliability would need to be done. 


This first part of ESINQ calibration supported the assessment of the calibration 
status of all ESINQ effects. Because I made earlier assessments for the values of the 
ESINQ effects, my only requirement during ESINQ calibration was to ensure that all 
ESINQ factors had been calibrated. To do this validation, I assessed the calibration status 
of the two remaining ESINQ surrogate factors following the execution of the EEMM, 
which included Td and Pc. Of these, both Td and Pc were calibrated during agent and 
ExtDep calibration. Thus, both ESINQ factors had been calibrated implicitly during the 
previous calibration processes, and aceording to the IMDP, no further action was needed. 
Had there been an ESINQ surrogate factor that had not been previously calibrated, the 
IMDP would have directed me to calibrate that factor similarly to previous calibration 
steps, to include bounding of the factor ranges, DOE, analysis, and verification. An 
example of the full process can be found in the IMDP. With calibration complete, I 
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recorded the final baseline and ExtDep calibration values for use during the build of the 
final model, as shown in Table 49. 


Table 49. Final Calibration Values 


Base Line ( 0% Ext Cont) Expanded ( 100% Ext Cor 


Cone 

10 

65 


N/C 

N/C 

Ph 

0.15 

0.345 


N/C 

N/C 

RoF 

12 

26 


N/C 

N/C 

Speed 

30 

0 


N/C 

N/C 

Pc 

0.350 

0.300 


N/C 

0.230 

Td 

8 

10 


N/C 

12 


These calibration values would be used in the model to capture the specific 
mission, agent, external dependencies, and ESINQ effects (0% impact) while ensuring 
consistent outcomes between the IRCPAT and the model, across all potential mission sets 
and FRs. The only values to be modified after this point are the ESINQ effects, and from 
that modification I would be able to quantify the impacts that they would have on the 
Blue Pv, which will be discussed in the next section. 

The end state of the calibration steps of the IMDP produced a model that could 
account for the advantageous of the specific mission, agent attributes, external 
dependencies, and ESINQ effects without introducing any new factors. By simply 
calibrating agent attributes in the model that already exist; it was possible to implicitly 
model effects of interest without inducing additional model complexity. This is best 
described by comparing all three verification plots, as seen in Figure 71. 
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Figure 71. IMDP Calibration Verification Curves 


As shown in Figure 71, as I progressed through the calibration process, I 
gradually increased the capacity of the model to represent the actual OF. As this accuracy 
was increased, a better understanding of the OF was gained, and thus, a better assessment 
of RCP was achieved. As shown in Figure 71, as I progress through mission, agent, and 
finally external dependencies calibration, the number of blue tanks needed to achieve the 
same Py decreased. Thus, through implicit modeling, I was able to develop a more 
accurate model and IRCPAT that were capable of accounting for factors that were 
previously ignored or aggregated. This reduced the overestimation of combat forces 
needed to achieve a specific Py, which has become a systemic issue with modern 
operational planning. For example, in this example and only considering external 
dependencies, I was able to determine that the FRC, which is the primary operational 
planning tool of the Army, was overestimating the number of tanks required to achieve a 
specific Py by roughly 11%. But this was just the beginning, now that I had established a 
stable reference point between the model and the IRCPAT, I was able to introduce 
FSINQ effects into the model and provide a quantifiable assessment of their impacts on 
metrics of combat effectiveness, which would further refine the RCP of Blue forces. 
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2 . 


Part B: Model Development 


Part B of the IMDP expands the capabilities of current MDPs and MBSE analysis 
methodologies by providing users an improved methodology for producing more accurate 
models using the calibrated data from part A. The purpose of model development is to 
finalize the implementation of the simplified model used during part A, and then to 
expand that model to highlight the impacts from the ESINQ factors of interest on metrics 
of operational effectiveness. Part B of the IMDP does this through the formalized 
realization and manipulation of the model, with the goal of quantifying the impacts from 
ESINQ effects on operational effectiveness. The four sub parts of part B will be executed 
in order to address the scenario described at the begging of this chapter, beginning with 
model development, where we finalize the model, expand the model to include the 
ESINQ effects, then conduct DOE, analysis, and then quantifying the ESINQ effects. 

a. Part Bla: Model Development 

The intent of this step was to finalize the development of the model to mirror the 
full operational scenario of interest, accounting for all requirements, specifically the ones 
not addressed during the build of the simplified model in part A. Eor this demonstration, 
the ignored aspects of the model focused on the additional forces as outlined in the OC 
established during part Ale, which included the artillery for both forces. To update the 
model to include these agents, I executed the 2-step framework shown in Table 50 for 
each additional agent type. 


Table 50. IMDP: Part Bla 


1 

Model Development: Iterative model development, analysis, and verification. 

- Model Development. 

2 

Implicit Calibration. 


- DOE. 


- Implicit Calibration Analysis and verification. 

3 

Model analysis and verification. 

- Model analysis: collect summary statistics. 


- Model verification. 
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During the first iteration of this step, I finalized the development of the simplified 
model and then updated it to include the opposing artillery systems, which were the next 
most significant force pair as noted in the IRCPAT established in part A3al. These 
systems included six M260A (Multiple Launch Rocket Systems) for the Blue force, and 
six 9P140 (220mm) for the Red force. I implemented them as outlined by my OC, and 
used the most accurate information available on their individual performances based on 
open sources as well as the FA-SWN. The only exception was that I used the baseline 
Mission, Agent, ExtDep, and ESINQ factor setting established during calibration and 
modified artillery speed to account for artillery employment techniques, which enforces 
fairly strict movement protocols. Next, I established all doctrinal communications and 
sensor links necessary for the artillery to perform its mission. Eollowing my update, I 
iteratively developed the opposing artillery systems within the model by modifying Blue 
burst radius until a 100 replication run of the model resulted in a mean Blue Py that was 
+/- 5% from the IRCPAT; the analysis of which can be seen in Eigure 72. 



Blue Pv _ 67 



Blue Tank Red Tanks Blue HQ 

Red HQ 

Blue Arty 

Red Arty 

Killer totals 

Blue Tanks 

0.0 

16.8 

0.0 

0.0 

0.0 

0.9 

17.7 

Red Tanks 

32.2 

0.0 

0.0 

0.0 

0.0 

0.0 

32J 

Blue HQ 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

Red HQ 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

Blue Aftv 

0.0 

0.7 

0.0 

0.0 

0.0 

4.7 

5.3 

Red Arty 

2.2 

0.0 

0.0 

0.0 

0.0 

0.0 

2.2 

Victim totals 

34.4 

17.4 

0.0 

0.0 

0.0 

5.6 

57.4 


Eigure 72. IMDP: Part Bla-1 (Artillery Implicit Calibration) 


As shown in Eigure 72, the mean Blue Py was 67.0%, just 1.5% from the 65.5% 
predicted by the IRCPAT, and well within the +/- 5% needed for initial verification. 
Eikewise, the artillery response in the killer-victim spreadsheet was appropriate for the 
opposing artillery missions, with Red artillery focusing on delaying approaching tanks 
and the Blue artillery focusing on the destroying enemy artillery. Thus, from my 
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perspective the behavior of the artillery in the model seemed appropriate, and thus, I 
considered the models initial verification for artillery complete, and I moved on to 
implicit calibration. 

I started implicit calibration by identifying the surrogate factors I would need to 
modify. Unlike previous calibration steps, implicit calibration was simpler because I took 
advantage of the fact that the majority of the combat power of the model had already 
been accounted for during tank calibration. Thus, the addition of other forces, like the 
artillery, would have far less impact on the outcomes of the model, which allowed me to 
calibrate the artillery through the use of just a few surrogate factors while maintaining 
synchronization with the IRCPAT. As the IMDP directs, I kept this simple and chose just 
three factors for the re-calibration of the model to account for the addition of the Blue and 
Red artillery. These factors included the Ph at max range for Blue and Red, as well as 
concealment for Red in the firing state. Next, to account for the change in FR due to the 
addition of artillery, I updated the FR ranges to maintain the 40% to 95% Py calibration 
bounding, which yielded a slight increase in the number of tanks (37-56) due to the 
reduction in the overall RCP of the Blue force following the addition of the Artillery. 
With FR range established and implicit calibration factors identified, the next step was to 
establish the bounds of the DOE, which I did by using the DOE range tool provided by 
the IMDP; the results of which can be seen in Table 51. 


Table 51. Implicit Calibration DOE Tool (Artillery) 


Factor 

Agent 

Attribute 

Cal 

Modifier 

DOE Low 

DOE High 

ItBlue Tanks 

N/A 

N/A 

37 

56 

BlueArtyPh @ 

0.60 

0.12 

0.49 

0.64 

RedArtyPh @ [ 

0.55 

0.15 

0.51 

0.67 

RedArtyConc { 

80 

0.15 

74 

98 


As shown in Table 51, after I transferred the estimated Phof the opposing systems 

into the agent attribute column, I calculate the contributions of each force with respect to 

the RCP of the overall force, and then applied this percentage to the calibration modifier 

column. Using this input, the tool calculated a reasonable DOE range for each of the 
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three agent factors. With the implicit calibration ranges established, I was able to execute 
a multi-factor DOE following the same steps as outlined in part A3b-e (Calibration), 
which for this demonstration resulted in an eleven-factor NOLH design (33DPs). After I 
stacked the design eleven times and replicated 250 times, the resulting design had a total 
of 90,750 DPs. Once the output was returned, I conducted analysis to identify the values 
for the three agent attributes that ensured the model was still calibrated {+!- 5%) to the 
IRCPAT. To do this, I began as I had during part A, by first updating the FR to account 
for the FE of the artillery, and then verified that the solution space was saturated. The 
results of this analysis can be seen in Figure 73. 



Figure 73. Artillery Implicit Calibration (Solution Space Saturation) 

Based on the definition of a saturated solution space described in Chapter II and 
III, my analysis of the implicit calibration DOE results suggested that the solution space 
is not saturated, deviating by just 2.3% from the desired 60%/40% split. Though the 
IMDP directs further modification to the DOE ranges to achieve a less than 10% 
deviation in distribution about the expected victory curve, because of the tightness of the 
DPs about the curve, I am confident that this saturation will be sufficient enough to 
proceed, and as long as no issues arise during the one factor verification this 
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simplification should be acceptable. Following this test I moved onto regression analysis 
using JMP, the results of which can be seen in Figure 74. 



'' Response Mean(bWin) 
Actual by Predicted Plot 
1 


04 0.5 0.6 0.7 06 0.9 1 

Mean(bWin) Predicted P<.0001 RSq=0.97 
RMSE=0.0247 


1 Summary of Fit 

RSquare 

0.97316 

RSquare Adj 

0.972049 

Root Mean Square Error 

0.024735 

Mean of Response 

0.770754 

Observations (or Sum Wats) 

353 


1 Analysis of Variance 

Source 

DF 

Sum of 
Squares 

Mean Square 

F Ratio 

Model 

14 

7.4979084 

0.535565 

875.3806 

Error 

338 

0.2067911 

0.000612 

Prob > F 

C. Total 

352 

7.7046996 


- .0001* 


Figure 74. Artillery Implicit Calibration (Regression Analysis) 


2 

The analysis suggests that the model has an excellent fit, with an adjusted R of 
0.97, and the ANOVA showing high confidence in its validity. Red concealment was 
determined to be insignificant when compared to FR, which dominates the response, and 
thus would not be modified from their base values during the implicit calibration of the 
artillery. While both Blue and Red Ph were significant, they were only marginally so, 
with Blue Ph being the dominate factor of the two. Thus, to simplify the calibration 
process, I will focus on the modification of Blue Ph. TSE was then conducted using JMP 
contour profilers, which can be seen in Figure 75. 
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Horii Vert Factor Current X 


o 

BlueArtyPh 

< 

=> 0.57 

o 

O RedArtyPh 


=3 0.55 

o 

O RedArtyConc 

<=0-- - 

=3 80 

® 

O FR(Arty) 

O = 

=> 2.5 


Response 

Contour Current Y Lo Limit Hi LimK 

— 



, 0.5460144 

— 



. 0.5061866 

— 

VarFromExp <■— 

. 0.0398278 0 0.04 



2.5 2.75 3 3.25 

FR(Arty) 


Figure 75. Artillery Implieit Calibration (Contour Profiler) 


As shown in Figure 75, while maintaining Red Ph and Red Coneealment at their 
base values, I was able to explore the tradespaee and identify a Blue Ph that minimized 
variation from the expeeted vietory eurve, whieh ealled for modifying Blue Ph from its 
base value of 0.60 to 0.57. Thus, with these settings, the model, which now included 
opposing artillery forces, should maintain its calibration with the IRCPAT (+/- 5%) 
across all FRs of interest (2.50 to 3.22), denoting the range between 50% and 90% Py. To 
verify, I executed a one factor DOE (1000 replications per DP) where the number of Blue 
tanks was varied from 37 to 56, the results of which are shown in Figure 76. 
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Figure 76. Artillery Implicit Calibration (One Factor Verification) 

Following implicit calibration of the artillery systems, my analysis of the one 
factor DOE results uncovered two artifacts that I was forced to address. First, was the fact 
that the mean Py curve was consistently above the expected victory curve. A better 
distribution of Py about the expected victory curve would have been more ideal. Second, 
there were three Py outliers, the first two were below the 2.50 FR, and thus ignored, while 
the third, seen at a FR of 2.53, failed to meet the +/- 5% variation from the expected 
victory curve required by the IMDP for calibration. Thus, the IMDP recommended re¬ 
calibration to address these issues, specifically by going back and decreasing Blue Arty 
Ph from 0.57 to 0.55 and re-doing the verification run. Nonetheless, because this one 
point was just outside the calibration range (5.02%), and because this was just a 
demonstration, I felt it was close enough to continue on without re-calibration, although 
the recommended changes to Blue Arty Ph were implemented. Thus, my first iteration 
was complete, and following the direction of the IMDP, I would repeated this process for 
the remainder of the forces noted on the IRCPAT until all forces had been implicitly 
calibrated and incorporated into the model. For this demonstration, I had no other system 
pairs to calibrate, and thus, no further implicit calibration was necessary. Once all 
additional forces described in the IRCPAT were implicit calibrated, I used the final 
implicit calibration data as a reference point, and recorded the baseline values for the 

ESINQ factors of each agent, as well as the meta-model describing the Py based on the 
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FR. This step established a baseline set of conditions from which all future expansions of 
the model could be measured to account for the impacts of the ESINQ factors of interest, 
as will be seen in part B2a. This data can be seen in Figure 77. 


Base Line (0% Ext Cont) | 

Attribute 


Cone 

10 

65 

Ph 

0.15 

0.345 

RoF 

12 

26 

Speed 

30 

0 

Pc 

0.350 

0.300 

Td 

8 

10 


Expanded (100% Ext Cont) 

Attribute 

HA 


Cone 



Ph 



RoF 



Speed 



Pc 


0.230 

Td 


12 


Implict Calibration (Arty) 1 

Attribute 


Cone 

70 

80 

Ph 

0.550 

0.550 

RoF 

10 

10 

Speed 

15 

15 

Pc 

NA 

NA 

Td 

NA 

NA 


' Fit Curve 
Model Comparison 

Model AIC< BIC SSE MSE RMSE R-Scfuare 

Lo9ist>cSP - -110.8731 -111.3603 0.0018201 0.0001213 0.01101» 0.99S4S47 

Plot 


2S 30 
fRtArty) 


' * Logistic 5P 
- Prediction Model 


(d-c) 


► Expj - a ■(FRCAHyj-b)]] 


a s Growth Rate 
b = Inflection Potni 
c s Asymptote 1 
d « Asymptote 2 
f = Power 
> Summary of FH 
^ Parameter Estimates 

Parameter Estimate 

Growth Rate -6.370U9 

Inflection Poirtt 2.226564 

Asymptote 1 1.0709371 

Asymptote 2 0-2469745 

Powg _ 0-247425 


Std Error Lower 9SX Upper 9SX 
23.356925 - 52.14926 39^08203 

0.7202588 0.8148828 3.6382453 

0.0874256 0.899586 1.2422882 

1.2181688 -2.140593 2 6345415 

1.02271» -1.757066 2.2519164 


Figure 77. ESINQ Expansion Reference Point 


This data served as a reference point for my manipulation of the model to observe 
the impacts from the ESINQ factors of interest. There was no need to record all possible 
agent attributes, because from this point forward, the only the ESINQ attributes of 
interest would be modified, which based on the EEMM, were Red Pc, and Red Td. After 
codifying the baseline statistics, I was able to move on to model expansion. 

b. Part B2a: Model Expansion 

The intent of this step is to expand the model to account for the ESINQ effects of 

interest identified during part A of the IMDP. While this was a fairly straight forward 
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process, requiring only minimal modifications, care was taken in how the modifications 
were made to ensure the process provided more than a subjective assessment. To do so, I 
started by importing the ESINQ effects of interest from part Alc/EEMM into the ESINQ 
bounding tool provided by the IMDP. Then, I identified the number of levels I wanted to 
represent the impacts from degraded ISR on operations. To keep this demonstration 
simple, I chose to use just five levels, representing the 0%, 25%, 50%, 75%, and 100% 
levels of severity. Then, using the values curves provided by the IMDP, I articulated to 
the best extent possible the expected form, or shape of the response curve, using all 
possible sources of quantifiable data. Typically, the form of the curve is better understood 
than the quantifiable impacts, which is why we start here. The scaling of these curves will 
come later during bounding. While the IMDP recommends the use of SMEs and other 
stakeholders to verify these curves, because I was acting as the SME for this 
demonstration, this verification process was unnecessary. The completed IMDP bounding 
tool is shown in Table 52. 


Table 52. IMDP: Part B2a-I (Initial ESINQ Impact Assessment) 


1 


Initial ESINQ Response Assessment. 

■ Import the linked Input factors from part Alc/EEMM into column 1 for the 1st ESINQ factor 
of interest. 

• Identify the number of levels of severity you want to represent, more is better, and will yield 
a better representation of the expected response, but for simple curves, it may not bee 
needed. This data will drive the creation of response meta-models used for instantiating the 
DOE. Create a Column for each of these, for this example, I used just 5 levels of severity as 
seen in Column 2. 

- Input expected impact (% change from base value) based on severity for each of the linked 
input factors into column 2. This should be informed to the largest extent possible by 
quantifiable sources of impact data. The key is to assess as accurately as possible the 
expected form, or shape of the response curve. Typically, the form of the curve is better 
understood than the quantifiable impacts, which is why we start here. The scaling of these 
curves will come later during bounding. Use the value curves to help support the impact 
assessment (the curve). This should be assessed by as many SME as possible, leading to an 
outcome that is acceptable by most. 


1 

2 

Linked Input 

ESINQ Factor Severity (Degraded ISR) 

Factors 

Extreme 

High 

Moderate 

Low 

None 


(100%) 

(75«) 

(50%) 

(25%) 

(0%) 

Ave TIme/Oet 

10.0% 

5.0% 

1.6% 

0.6% 

0% 

Pc 

4.CW 

3.0% 

2.05 

1.0% 

0% 


0 Because this demonstration was looking at the impact of degraded ISR on Blue tanks, in 
a small protracted fight, the immediate impacts on the force will be relatively small. 
Thus, the degradation to combat effectiveness Is likely small. For other forces, or with 
other ESINQ factors, these curves could be quite different, it is for this reason that 
these curves should be done for each agent-ESiNQ combination. 


^ 15.0% 

I 10.0% 

I « 5.0% 
I h 
E 0.0% 


5.0% 
S A.0% 
E 3 0% 

Z 2 . 0 % 
g 1.0% 

i 

a 


RedTd 



Extreme High Moderate Low None 

(100%) (75%) (50%) (25%) (0%) 

ESINQ Sever'tty 


Red Pc 



Extreme High Moderate Low None 

(100%) (75%) (50%) (25%) (0%) 

ESINQ Severity 
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As shown in Table 52,1 started by attempting to capture the expected impacts that 
degraded ISR would have on each of the two ESINQ surrogate factors for each of the five 
levels of severity. I used the curves to help shape the response of each factor based on my 
understanding of the effect, as well as the expected impacts. For example, for Td I 
assessed a maximum impact of 10%. I expected this impact to be small at first, gradually 
increase with severity, and then increasing polynomial as severity increased above 50%. 
Using the tool I modified the degradation values of Td at each level of severity until a 
curve matching my expectation was achieved. The modification of the Pc surrogate factor 
was executed similarly. 

With the initial ESINQ response assessment complete, I moved on to bounding 
the response of the ESINQ surrogate factors. Because the user will likely only have a 
basic understanding to approximate the expected impacts from ESINQ factors, an 
investigation into the power of the response to impact operational effectiveness is needed 
to bound the response to a region of feasibility. Using the value curves established 
previously is extremely subjective, and would likely induce more variability into the 
model outcomes than would be desired. This step helps refine the estimation of factor 
settings by bounding them through DOE. To do this, I started by assessing the maximum 
likely impact from ISR degradation on each of the surrogate factors compared to the 
contributions from external dependencies, which grounds the assessment of ESINQ 
impacts to something that has already been accounted for and calibrated in the model. For 
this example, I assessed this impact to be 20% of the total contribution of external 
dependencies. Using the response bounding tool, I then calculated the total estimated 
range of potential degradation across the range of levels of severity, and applied these 
adjustment factors to the model base values to establish the ESINQ bounding DOE 
ranges. This step and associated tools can be seen in Table 53. 
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Table 53. IMDP: Part B2a-2 (Bounding the ESINQ Response) 


2 Bounding the Response. 

- Because the user will likely only have a basic approximation of the expected impacts from 
ESINQ factors...which is why they are ESINQ, an investigation into the power of the response 
to impact operational effectiveness is needed to bound the response to a region of 
feasibility. Using the values from above is extremely subjective, a best guess, and would 
likely induce more variability into the model outcomes than would be desired. This step 
helps refine the estimation of factor settings by bounding them through DOE. 

- First select a base ESINQ impact value for each factor. To do this, make an assessment of 
the likely maximum % impact that the ESINQ effect would have compared to the 
contributions from External dependencies. This grounds the assessment of ESINQ impacts 
on something that has already been accounted for and calibrated in the model. 

0 For this example, I assess that the maximum of degraded ISR to be roughly 20% of the 
impacts from external dependencies seen in part A3d, which was captured in the 
following factors settings: 



0% Contribution 

100% Contribution 

Delta 


Red Pc 

Red Td 

Red Pc 

Red Td 

Red Pc 

Red Td 

0.300 

10 

0.230 

12 

-0.070 

2.000 


Maximum 

Impact 

0.20 







0 Thus, the maximum expected impact from ESINQ effects is: 



Estimated Impact 



Red Pc 

RedTd 



-0.014 

0.400 




Import the baseline ESINQ factor settings from part A3d, to include the modification to 
account for 100% external dependencies. Import just the ESINQ effects identified in 
Alc/EEMM, using the values from part A3d. Recall, a degradation to Blue forces in this 
example is shown through a improvement to Red capabilities. 


Build a multifactor DQE across the range of impacts established above, using the FR range 
from Bla. Round where necessary in the direction of an expanded range. For a better 
model fit, apply the range to both sides of the base value, failure to do this risks a flawed fit 
which can induce anomalies into data analysis. 


DPs 

Factors 

Low 

Base 

High 

1 

# Blue Tanks 

37 

42 

56 

2 

Red Td (sec) 

11.0 

12.0 

13.0 

3 

Red Pc 

0.216 

0.230 

0.244 


Following the establishment of the DOE ranges, I used a six factor 2"‘^ order 
NOEH design to produce the data necessary to bound the ESINQ factors. I stacked the 
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design six times and replicated it 200 times, for a total simulation run size of 84,000. The 
analysis of this data focused on establishing a set of surrogate values where the maximum 
impact to operational effectiveness was close to the estimated maximum impact from the 
ESINQ effects used to compare external dependencies. In this example that was roughly 
20%. Additionally, because we established that external dependencies contributed around 
19% of the RCP of the Blue force according to the FA-SWN, I was looking for a 
maximum impact to RCP from ESINQ effects of roughly 4%, or about a 7% impact to Py. 
This provides us the ability to bound the maximum impact from ISR degradation using 
two metrics, maximum impact to the surrogate factors as well as the maximum impact to 
Py based on the contributions from external dependencies. Both of these curves can be 
seen in Figure 78. 



Figure 78. Part B2a-3 (ESINQ Response Bounding) 


As shown in Figure 78, each of the 420 DPs from the DOE was distributed about 
the meta-model curve developed in part Bla, which represents the potential outcomes 
from the fully calibrated operational model. The green curve was the lower bound, 
representing the maximum possible degradation based on the impacts to RCP, which was 
7%. The purple curve representing the upper bound, which highlighted the maximum 
impact due to surrogate factor degradation determined through the DOE, which for this 
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example resulted in roughly a 4.25% degradation. It is the responsibility of the user to 
fully investigate the difference between these two bounding curves, and based on that 
analysis; make a decision on which they will use. With this in mind, there are three 
general options in which the user can take to make this determination. First, the user can 
choose to use the estimation based on the impacts to RCP (the green curve), which 
requires the expansion of the DOE ranges and repeated DOE until the results capture that 
curve. Second, the user can choose to use the DOE results (the purple curve), which 
estimates the maximum impact based on impact to surrogate factors. Third, the user can 
chose a hybrid, where the estimation is made based on the relative confidence in both of 
the bounding estimates, and thus, the resulting curve will fall somewhere in between. 
This option would also require expanding the DOE ranges to capture the desired curve, 
and will likely provide a more neutral outcome. Eor this example, my confident in the 
assessment of Td and Pc as primary surrogates and the relative impact to each of these due 
to ISR degradation was fairly high. Thus, I used this curve to represent the maximum 
impact rather than the more generalized 20% maximum degradation at the lower bound. I 
believed that this assessment was more realistic given the scenario for this demonstration, 
and under scrutiny, it seemed to meet face validation. Eollowing my analysis, the 
following surrogate factor settings provided the most robust option for representing the 
desired effects while maintaining consistency with the surrogate meta-model, the contour 
profiler of which can be seen in Eigure 79. 
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Figure 79. ESINQ Surrogate Bounding Estimates 

To make the determination of surrogate factor settings, I started by modifying the 
most significant factor other than ER, which in this case was Red Td, until the most stable 
response across the potential ERs is achieved. Then, I systematically reduced the range of 
the mean from the maximum surrogate value and modified Red Pc until the minimum 
variance was achieved while ensuring that the entire ER region of interest was still 
feasible. This resulted in a Red Td of 11.1 and a Red Pc of 0.238, with a variance from the 
maximum surrogate curve between 0.4% to 1% Py. With the bounding of ESINQ 
responses complete, and a set of ESINQ surrogate factor settings established that would 
represent the maximum impact from ISR degradation, I moved on to modifying the 
baseline model factor settings to represent the expected impacts for the remaining levels 
of severity. I accomplished this by using the supplied tool, starting by establishing the 
upper bound of degradation by importing the ESINQ surrogate factor settings identified 
in Eigure 79 into the 100% degradation column of the tool. Then, to establish the lower 
bound, I imported the base line value settings from part Bla into the 0% degradation 
column. Eollowing these inputs, the expected values for each of the ESINQ surrogate 
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factors were calculated for each intermediate level of severity. This step and the 
instantiated tool can be seen in Table 54. 


Table 54. IMDP: Part B2a-4 (ESINQ Response Curve Meta-models) 


Modify baseline model factor settings. 

Following analysis and TSE, record the surrogate factor bounded ranges here under the 
1009( degradation, as well as the baseline values under 0% degradation. 


Linked input 
Factors 

ESINQ Factor Severity (Degraded iSR 


Extreme 

(100%) 

High 

(75%) 

Moderate 

(50%) 

LOW 

(25%) 

Nor>e 

(0%) 

Red Pc 

U.JiB 

0.236 

0.234 

0.232 

RedTd 

11.1 

11.55 

11.86 

11.95 

12 


Note: Because i am using Red values here to represent impacts (degradation) to Blue 
forces, I need the reciprocal response from what Is expected. For example, to 
Implement the desired degradation In Blue Pc, i need to improve Red Pc with each 
increase in severity. 

Verify the tool ht the data to the curves by comparing the original impact curves to the 
ESINQ impact curves. 

Create rrveta-models of each surrogate factor that accepts level of severity from O-lOO, 
and outputs the expected value for that surrogate, record them here. For this i used 
JMP, based on the data table above. 


Red Pc 


0 + 0.00008*Severity 


-1.25053+(0.0221522+1.25053)/(l+EXP(0.043144*(SeverttY- 

156.76503»)*15.628862 


These meta-models will be used to Inform the DOE in the next step based on the level of 
severity. The meta-models are captured below, and a test Is provided to investigate the 
response and for trouble shooting. 


1 12.0% 

J 10.0% 

1 8.0% 

£ 6.0% 

Z 4.0% 

I :.o» 

£ 0.0% 

RedTd 

_ 


X 



* m 

ESINQ Severity 

1 S.0% 

■S 4.0% 

1 3.0% 

a 1 . 0 % 

1 0.0% 

V 

Red Pc 






ESINQ Severity 


RedTd (Meta-Model) 



i 0.240 

t 

^ 0.238 
i 0.236 
5 0 234 
f 0.232 
» 0.230 


gamQSevertty 


Red Pc (Meta-Model) 




Following the development of the meta-model curves, I inspected them for 
conformity with the original curves, ensuring the generate shape of the response was 
consistent, which for this example they were. Thus, following this verification, I 
developed meta-models for each of the surrogate factor responses that would accept the 
level of severity and output the expected adjustment factor for both red Td and Pc, which 
will be used to inform the DOE in the next step. At this point, the model could have been 
updated for any given combination of severity, and following a verification run, analysis 
would have highlighted the variation in the Pv due to the impacts from degraded ISR for 
that specific combination of FR and severity. While this improved model would have 
been useful in itself, because I was interested in developing meta-models that could 
represent the impact of degraded ISR across all levels of severity, it was unnecessary; all 
combos were to be explored during the subsequent DOE. 


c. Part B2b: DOE 

The intent of this DOE was to produce sufficient data to enable detailed analysis 

in the next step, where impacts to Blue Py based on FR and ISR degradation would be 

quantified, and a meta-model of the response codified. Following the direction of the 
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IMDP, I started by establishing the bounds for the FR based on the directionality of the 
expected response of the ESINQ effects. This was important to ensure the DOE retained 
enough flexibility in ER to maintain a Pv range between 40% and 95%. In this 
demonstration I was concerned with assessing the impact of degraded ISR on Blue Py, 
which was expected to degrade Blue RCP. Thus, the upper bound was extended by 50% 
of the PR range from part B2a while fixing the lower bound, which resulted in a PR range 
of 37-65. Next, I expand the upper bound of severity by 25% to allow more flexibility in 
the design to account for degradation uncertainty. The steps for this step, as well as the 
DOE range tool can be seen in Table 55. 


Table 55. IMDP: Part B2b-I (DOE Range Bounding) 


1 Build the DOE design starting with the FR and levels of severity. Because we are unsure of the 
response, and want to give the user more flexibility, we will use a wider range of FRs as well as 
severity. 

- Special care must be taken when establishing the bounds for the FR based on the impact of 
the ESINQ effects of interest. We must ensure the DOE has enough flexibility in FR to 
maintain a Pv range between 40% and 95%. Thus, if the ESI NQ effect will improve RCP, the 
lower bound must be extended while fixing the upper bound. If the ESINQ effect will reduce 
RCP, as in this example, then the opposite is true, and we fix the lower bound and extend 
the upper bound. As a general rule; 

0 If extending the lower bound, extend it by 50% of the difference between the bounds 
used during part B2a. For this example, that was 37-56 tanks, so the lower bound would 
be decreased by 9, establishing a new lower bounds of 28. 

0 If extending the upper bound (as we are in this example) do the same, just modifying the 
upper bound by 9, establishing a new upper bounds of 65. 

- Expand the upper bounds of severity as much as the user is comfortable with, which will be 
based on the accuracy of the meta-models developed in the previous step to capture the 
response of each surrogate factor. For this example, I choose to expand the range by 25%. 
This allows more flexibility in the tool to account for degradation uncertainty. 




DPs 

Factors 

Low 

Expected 

High 



1 

# Blue Tanks 

37 


65 

2 

Severity 

0 


125 

3 

Red Td (sec) 

Calculated for each Dip using meUk 
models based on severity 



4 

Red Pc 


0 At this point the DOE will only have 2 factors, but will be expanded to include the ESINQ 
surrogates after the design is built, which will be calculated using the meta-models 
developed during part B2a. Some considerations for the design: 

- Use a larger design then needed, saturation is key with such a wide range of 
severity. 

- For this example, I used an 7 x factor 2nd Order NOLH, which has 125 DPs, and can 
be seen to the right. 
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Following the establishment of the DOE ranges for these two factors, a order 
NOLH design was developed. As directed by the IMDP, to ensure adequate saturation of 
the solution space I used a seven factor design, which resulted in 125 DPs. Then, to 
further saturate the design I stacked the design seven times, resulting in an 875 DP 
design. This two factor design was then copied to a clean spreadsheet, where I added two 
additional columns, one each for Red Tj and Pc, which were calculated using the meta¬ 
models developed during part B2a based on the values of FR and Severity at each DP. 
Finally, based on the models input requirements, some further modification to the data 
was necessary. For MANA, this required me to set #BlueTanks to discrete values, modify 
Td to increments of 0.1, and add a scaling factor for both Td and Pc. A partial snap shot of 
the final 875 DP four factor design can be seen in Table 56. 


Table 56. IMDP: Part B2b (Consolidated DOE) 


BlueTank 

Severity 

Red Td 

Red Pc 

56 

41.4 

119000 

2330 

63 

121.9 

108000 

2400 

55 

51.3 

118000 

2340 

47 

93.9 

112000 

2380 

38 

112 

109000 

2390 

61 

86.8 

114000 

2370 

54 

71.7 

116000 

2360 

52 

40.6 

119000 

2330 

62 

17.9 

120000 

2310 

47 

115.8 

109000 

2390 

49 

71.5 

116000 

2360 

57 

102.7 

110000 

2380 

63 

17.2 

120000 

2310 

50 

78.7 

115000 

2360 

46 

82.4 

114000 

2370 

53 

67.5 

117000 

2350 

65 

12.2 

120000 

2310 

45 

81.1 

115000 

2360 

58 

55.6 

118000 

2340 

50 

85.6 

114000 

2370 

39 

37.2 

119000 

2330 

44 

75.7 

115000 

2360 

38 

68.6 

117000 

2350 

49 

26.1 

120000 

2320 
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It is important to note that while Td and Pc served a critical role as surrogate 
factors for ISR degradation in the model, following the DOE, they were no longer 
needed. During analysis, the impacts to Blue Py were linked directly with ISR 
degradation severity. Following the completion of the design, the design .csv, the model 
.xml, and the study .xml were uploaded to the NPS advanced computing cluster, and after 
each DP was replicated 400 times, resulted in a total of 350,000 simulation runs. After 
the output data was returned and verified, I was able to move on to the next and final step 
of the IMDP. 

d. Part B2c: Quantifying ESINQ Effects (Analysis) 

The contribution of this work was to provide the means (the IMDP) to develop an 
ESINQ enable model, which was technically completed following during part B2a. 
However, we have yet to quantify the impacts from ESINQ effects in any meaningful 
way. The intent of this final step was to conduct a final iteration of analysis of the output 
data from the large scale DOE executed in the previous step to establish a mathematical 
representation of the impacts from degraded ISR on the model, specifically the impact to 
Blue Py. As directed by the IMDP, I started by first importing the data into IMP and 
conducting data formatting. In this first step I inspected the data for completeness, 
translated any modified factors back into their original form, add a column for Py, and 
added a column for FR based on the equation used in part Bla. Following the import and 
organization of the data, I collapsed it using a summary table for severity, FR, Red Td, 
and Red Pc, while producing the mean of the Blue Py. Then, I conducted regression 
analysis on the data to fit the model. For this demonstration, I used a standard least 
squares regression to conduct effects screening. Specifically, I fit severity and FR using a 
degree factorial and a 2"‘^ degree polynomial to construct my model effects, then used 
mean Py as my role variable. Following regression, I conducted analysis starting with the 
fit, as summarized in Figure 80. 
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RSquare 

0.987888 

RSquare Adj 

0.987817 

Root Mean Square Error 

0.017648 

Mean of Response 

0.825656 

Observations (or Sum Wgts) 

858 


Figure 80. IMDP: Part B2c (Summary of Fit) 


2 

As shown in Figure 80, the model fit had an adjusted R of 0.988, denoting a very 
good fit between the data and the resulting meta-model. This was far better than I was 
expecting for a combat model, which due to their highly stochastic nature, often have 
extremely wide ranges of variability, which typically reduce the overall fit. To investigate 
any such variability, analysis of variance was then conducted to gain a better 
understanding of the model, the results of which can be seen in Figure 81. 


Source 

DF 

Sum of 
Squares 

Mean Square 

F Ratio 

Model 

5 

21.644337 

4.32887 

13898.69 

Error 

852 

0.265363 

0.00031 

Prob > F 

C. Total 

857 

21.909700 


<.0001' 


Figure 81. IMDP: Part B2c (Analysis of Variance) 


The analysis shows that the model accounts for nearly 99% of the observed 
variability, with roughly 1% of the variability attributed to error. The models ability to 
capture the majority of the variability, coupled with the relatively insignificant mean 
square of the error component, results in an extremely large F Ratio. Knowing that the 
variability of combat outcomes can approach 100% as the forces approach parity, I am 
encouraged with the results, which likely derive from the high number of degrees of 
freedom during the analysis. The Prob > F value of less than 0.0001 is also encouraging, 
indicating a high probability that the model is in fact capturing the majority of the 
variability. Following analysis of variance, I inspected the sorted parameter estimates to 
look for significance in the response and directionality. It is important to understand what 
factors are significant, the direction of their response, and the inter-dependence of the 


225 





factors to gain better insight into the funetion of the model. Figure 82 shows the sorted 
parameter estimates. 


Term 

FR 

(FR-3.20929)*(FR-320929) 

Severity 

(Severity-62.5446)*(FR-320929) 
(Severity-62.5446)*(Severity-62.5446) 


Estimate Std Error t Ratio Prob>|t| 

0.3234164 0.001299 249.04 ~ “ 

-0.262041 0.003158 -82.99 
-0.000257 1.651 e-5 -15.56 
0.0003078 0.000036 8.54 

-2.228e6 5.086e-7 -4.38 


noOl 
<. 0001 * 
<.o:: 
<.ooov 
< nool* 


Figure 82. IMDP: Part B2c (Sorted Parameter Estimates) 


As shown, FR dominates the response with regard to Py, which was expected, as 
well as its 2"‘^ order interaetion. Severity of ISR degradation also had a fairly signifieant 
impact, though less than FR. Of special interest is the significance of the two-way 
interaction between FR and severity, which denotes a non-linear interaction between the 
two and provides some useful insight into the model. Non-linearity in models often 
highlights potential opportunities or eoncerns that should be understood before moving 
forward. To investigate this non-linearity further, I used a contour profiler to perform 
TSE, which can be seen in Eigure 83. 


226 













Figure 83. IMDP: Part B2c (Contour Profiler) 


To execute TSE I began by manipulating FR and the level of severity to gain 
insight into the impacts that degraded ISR had on the Py of the Blue force. What I found 
is that the impact of severity on Py is indeed non-linear, having a much greater impact 
when the FR is lower. Intuitively, this makes sense. Due to the fact that FR dominates the 
response, and that the impacts of degraded ISR on a relatively short tank battle is likely 
very small, impacts from ESINQ effects can often lose significance during battles where 
the Blue force enjoys a large FR. A key observation from the profiler is that while a 
greater than 80% Py was possible at a FR of 2.96, once ISR degradation is considered this 
was no longer the case. At 100% severity of degradation, the profiler suggest that we 
would need at least a FR of 3.03 to achieve a greater than 80% Py, highlighting a -0.07 
(2.36%) degradation in RCP. If the FR was not adjusted to mitigate the impact from ISR 
degradation, a -2.78% decrease in Py was incurred, which quantifiably captures the 
impact from degraded ISR on metrics of operational effectiveness. Thus, the primary end 
state of this dissertation has been realized. Yet, to provide more utility to the user, who 
may not be comfortable with or have access to IMP, the prediction expression from the 
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analysis was used to create a meta-model of the Pv based on the FR and ISR degradation 
severity, which can be seen in Figure 84. 


-0.1368213406751 
+ -0.0002569378203* Severity 
+ 0.32341636103974* FR 

+ (Severity - 62.5446386946387) * ((FR - 3.20928852593681) * 0.00030782503336) 

+ (Severity - 62.5446386946387) * ((Severity - 62.5446386946387) * -0.000002228191) 
+ (FR - 3.20928852593681) * ((FR - 3.20928852593681) * -0.2620413962063) 


Figure 84. IMDP: Part B2c (Meta-Model for Impacts from Degrade ISR) 

This meta-model provided me a mathematical representation of the impacts of 
degraded ISR on the outcomes of the model, specifically the Pv of the Blue force, based 
on the FR and the level of severity. Using this meta-model, I created a tool that could 
take the expected level of severity, FR, and FE (part of FR) as inputs and returned the 
expected impact on Blue Py and RCP from degraded ISR. This tool can be seen in Table 
57. 


Table 57. IMDP: Part B2c-2 (Meta-Model Testing Tool) 


Force Ratio 

2.96 

Blue Expected Pv 

80.03% 

ESINQ Severity 

100 

Blue Pv (ESINQ) 

77.25% 

FE 

635.76 

Blue Impact (Pv) 

-2.78% 

Blue Impact (RCP) 

-11.44 


Using this tool, I tested the meta-model thoroughly to ensure it was functioning 
correctly and that it provided the expected outcomes based on the analysis conducted in 
its development. With testing complete, the meta-model could be used to improve other 
tools - a secondary contribution of this work that will be discussed and demonstrated in 
Chapter V: Application. 
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C. SUMMARY 

This chapter presented a fully executed IMDP. This demonstration, and its 
associated products, served to illustrate the overall applicability of the IMDP to quantify 
the impacts of ESINQ effects of interest on metrics of operational effectiveness. As 
advances in technology and changing doctrine continue to enforce strategies that pass 
more of a systems internal tasks to ESINQ sources of combat power, the presence and 
impact of these effects can no longer be ignored and/or aggregated the models referent; 
they must be addressed and accounted for separately in the model. By providing a 
methodology in which users can quantify ESINQ effects and implement them in their 
models, the IMDP fills known gaps and expands the body of knowledge with regard to 
M&S and traditional MDPs. The meta-models that are the end state of the IMDP could be 
used to improve not only the model, but other operational and acquisition support tools as 
well. These improved tools would then provide a more accurate representation of the 
system with respect to the OE, and if used in support of decision makers, could improve 
the quality of their decisions. While the potential use for the meta-models generated 
through the use of the IMDP is nearly limitless. Chapter V: Application will cover just 
two such uses, describing the implementation of the meta-models to improve an 
operational support tool as well as an acquisitions support tool. 
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V. APPLICATION 


The primary contribution from the execution of the IMDP was the development of 
meta-models that can describe the impacts of ESINQ effects on metrics of operational 
effectiveness. These meta-models have a nearly limitless capacity to inform and to 
improve other tools. In fact, the application of the outcomes of the IMDP to improve 
current operational and acquisitions decision support tools was a key consideration in the 
focus of this dissertation. Although the application of the meta-models is outside the 
scope of the IMDP, the author would be remiss if some of the potential applications of 
the meta-models were not demonstrated. Thus, the purpose of this chapter is to apply the 
outcomes from the execution of the IMDP to modem problems. Because of the nearly 
limitless potential for these meta-models, this chapter will not provide any specifics 
regarding how to apply the outcomes. Rather, it will provide a description of how the 
author applied the meta-models to address the gaps identified earlier in this dissertation, 
specifically by improving of operational and acquisitions decision support tools. 

A. OPERATIONAL PLANNING SUPPORT TOOL 

Historically, most military plans are developed through detailed assessment of 
friendly forces, the expected opposing force, and the OE, supported by intelligence, 
higher guidance, and the experience and knowledge of staff and decision makers. As 
technology has advanced, so have the tools the military uses to support its operational 
planning, yet the majority of the force assessment and allocation of forces has remained 
the responsibility of staffs and leaders. While operational planning support tools play an 
informative role in this process, they are typically only used for validating other more 
subjective, human centric decisions and assessment processes. A contributing factor to 
this inefficiency is the inability of modem tools to provide anything other than the 
simplest of comparisons between forces, offering only a rough assessment of RCP. The 
ERC is a perfect example to highlight the inadequacy of current support tools; an 
instantiated version of which can be seen in Eigure 85. 
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Force Ratios 

Friendly Forces 

Enemy Forces 

Number 

Strength 

Type 

FE 

Total 

Number 

Strength 

Type 

FE 

Total 

5 

100% 

Armor Bn (44 x M1A1) 

1 24 

6 20 

3 

60% 

Infanttv Bn (BMP-3) 

0 65 

117 

2 

100% 

Armor Bn (44 x M1A2) 

1 30 

260 

1 

60% 

Tank Bn (TB 40xT90) 

1 06 

064 

1 

100% 

155(SP) Bn (M109A6, 3x6KPa 

1 50 

1 50 

1 

60% 

2S7 Bn 

1 28 

0 77 


100% 




1 

60% 

AT Bn (12 X 2A45 & 6 x AT-5/1 

035 

021 


100% 





80% 





100% 





80% 





100% 





100% 





100% 





100% 





100% 





100% 





100% 





100% 




Friendly Force Equivalent 10.30 

Enemy Force Equivalent 2.78 

Ratio of Friendly to Enemy 

3.70:1 

Ratio of Enemy to Friendly 

0.27:1 

Hasty Attack 

10% 

<- Mission -> 

<- Est. Losses -> 

Hasty Defense 

50% 




Instructions: 

1 Select type of unit from drop down list 

2 Input number and type If less than a whole 
unit use fractions (e g. 1 Btry = .33 Bns). 

3 Use comparison of force ratios (graph) and 
historical planning ratios 

4 To calculate damage to each unit, select th« 
friendly and enemy mission from the list. 

Remember: Relative force ratios do NOT 
necessarily indicate the chance for success 
for either force! 

Relative 
Combat 8 

Power 6 

4 

2 

0 


1 — 


Friendly Enemy 

Historical minimum planning ratios. 

FherxHv ntission 

Fnendly Enemy 

Position 

Deiay 

1 6 


Defend 

1 3 

Prepared or fortified 

Defend 

1 ; 2 5 

Hasty 

Attack 

3 ; 1 

Prepared or fortified 

Attack 

2 5 1 

Hastv 

Counterattack 

1 1 

Flank 


Figure 85. Force Ratio Calculator. Source: Craig (1999). 


As the primary tactical planning support tool used by the Army since its 
development by MAJ J. Craig in 1999, the FRC has a fairly wide distribution and is 
routinely used during planning for training and combat operations. While the FRC has 
been impactful in supporting operational planning, specifically with regard to force 
allocations, it has some significant shortcomings that have critically undermined its 
usability and effectiveness in supporting decision making. Of these shortcomings, there 
are five that should be addressed. First and foremost, the FRC is a simplified tool that 
aggregates combat into fairly simple quantifiable system versus system metrics, without 
accounting for additional sources of combat power like ESINQ effects. Second, as an 
aggregation tool, the FRC cannot account for the synergist impacts of modern combined 
arms operations on today’s battlefield. Third, because of its low resolution, the FRC is 
not able to accurately account for forces sizes below the BN level, which is no longer 
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realistic in the modern era of combat. Fourth, the FRC has had few updates over the past 
18 years, leaving the tool dated and unable to address many modern systems. These same 
issues are seen in the source data as well; the FA-SWN and the COFM are in dire need of 
updates or replacement. Finally, the FRC fails to maximize its utility for the user, missing 
opportunities to better support decision makers by making the tool more useful. To 
address some of these issues, the FRC was updated to create the IRCPAT, as seen in 
Figure 86, giving the tool more utility to modern day users. 


knproved Relative Combat Power Assessment Toot (IRCPAT) 
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Figure 86. IRCPAT 


233 























































































































As shown in Figure 86, the User Interface (UI) was significantly modified to 
provide the user with better manipulation of the tool as well as a more detailed 
assessment of RCP. This modification had four primary steps. First, the data tab was 
updated to provide finer resolution of the force sizes and to add new system types like the 
Stryker that were absent in the FRC. Next, the damage table was replaced using a meta¬ 
model, which accepted the FR and the mission set from the user to calculate the expected 
losses for each force. Because the FRC determined expected losses through a look-up 
table, it could only provide seven discrete values. By developing a continuous meta¬ 
model of this discrete response, the IRCPAT solved a significant issue with the FRC and 
can now more accurately assess expected damage across the entire range of potential 
FRs. Next, four victory curves were integrated into the IRCPAT to provide the user more 
functionality. These curves were loosely based on the works of (Helmbold 1969) and 
(National Research Council Committee on National Statistics 2003), which sought to 
establish historical linkages between FR and Py. By providing a range of victory curves 
for the user to choose from, a more flexible estimation of the Py can be achieved. This 
flexibility allows the user to modify the tool as needed based on their confidence in the 
assessment and the level of risk they are willing to accept. Finally, a set of TSE tools 
were added to the IRCPAT to support a better representation of the RCP trade space. 
These tools provide the user the ability to visualize the impacts that modification to the 
FR and the expected FER can have on the Py. With the IRCPAT created, it was now 
possible to further improve the tool by applying the outcomes of the IMDP from the 
previous chapters. 

1. Application of the IMDP Outcomes 

The first improvement offered by the IRCPAT was the delineation between 
internal and external sources of system combat power. During part A2b and A3d, the 
IMDP quantified the contribution from external sources of internal combat power, which 
can be degraded separately based on the status of those external systems. Thus, the 
IRCPAT was then updated to account for these contributions to RCP individually, as seen 
in Eigure 87. 
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Impfoved Relative Combat Power Assessment Tool (IRCPAT) 
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Figure 87. Expanded IRCPAT (External Dependencies) 


As shown in Eigure 87, the IRCPAT was updated to provide a more accurate 
assessment of system combat power by delineating between the two sources of internal 
combat power: power that is inherent in the system and power that is dependent on 
external resources. While the total potential PE of each system remains the same, it is 
now possible to degrade the external sources independently; the impact of which is 
dependent on the dependency of each system on those external sources. The dependency 
is greater for higher tier systems, and thus, the impact to the system PE is greater than a 
lower tier system. By accounting for this third area of combat power contributions, the 
IRCPAT can provide a much more realistic understanding of opposing systems. 
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The second improvement offered by the IRCPAT was the inclusion of ESINQ 
effects as additional contributors to the generation of RCP. During part B2c, the IMDP 
quantified the impacts of ESINQ effects of interest on metrics of operational 
effectiveness through the generation of meta-models. Eor this work, the ESINQ effect 
used for demonstration was degraded ISR: the meta-model for which accepted ER and 
level of severity and then provided the expected impact to RCP of the Blue force. 
Because the IRCPAT provides all the necessary inputs, the meta-models can be applied, 
which is why we (other than partial validation) maintain calibration with a reference tool. 
Eigure 88 shows the IRCPAT following the modification for ESINQ effects. 
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Eigure 88. Expanded IRCPAT (ESINQ Effects) 
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As shown in Figure 88, the IRCPAT was updated to account for the impacts from 
degraded ISR on the Blue force by including the meta-model generated at the completion 
of the IMDP. As before, the total potential FE of each system remains unchanged, but it 
is now possible to degrade the portion of Blue RCP based on the level of degradation 
severity. As the user assesses higher levels of severity, based on the levels (0% to 100%) 
used during part B2a of the IMDP, the degradation of Blue RCP increases. Additionally, 
because of the flexibility of the IRCPAT, other ESINQ effects can be included following 
the development of their meta-models. After this final improvement, the IRCPAT can 
now address all four of the potential contributors of combat power, proving a much more 
complete understanding of the actual OE. With the improved IRCPAT complete, a 
demonstration of its utility is appropriate. 

2. Demonstration 

To demonstrate the utility of the IRCPAT, it was instantiated for the example 
scenario used in Chapter IV. To do so, some user inputs were required, including status 
of external C4I, and ISR degradation. Eor this demonstration, external C4I was set at 
95% to denote the general status of external support sources, while the ISR degradation 
was set at 50%, denoting a decrease in access to space based ISR assets. The instantiated 
IRCPAT for this scenario can be seen in Eigure 89. 
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Friendly Force Equivalent 625.49 

Enemy Force Equivalent 235.64 

Ratio of Friendly to Enemy 

2 654:1 

Ratio of Enemy to Friendly 

0.377:1 


Hasty Attack 

<- Mission -> 

Hasty Defense 

8.1 1 16.8% 

<- Estimated Losses -> 

36 5% 1 9.5 

0515 

<- Estimated % Victory (Curve 1) -> 

0.485 

0.552 

<- Estimated % Victory (Curve 2) -> 

0448 

0.587 

<- Estimated % Victory (Curve 3) -> 

0413 

0.624 

<- Estimated % Victory (Curve 4) -> 

0.376 


Figure 89. IRCPAT (Application Demonstration) 


The IRCPAT has been updated to include the degraded ISR meta-model 
developed during Chapter IV, specifically formatted to provide a degradation factor with 
respect to RCP. As shown in Figure 89, the overall RCP of the Blue force has been 
reduced. When status of external C4I was set to 100% and ISR degradation set at 0%, the 
Blue RCP was 2.698:1 with a Blue Py of 65.5%. Following the modification of the 
external dependencies from 100% to 95%, the RCP dropped to 2.673:1 with a Blue Py of 
63.7%, a 1.8% decrease in Py. After accounting for the 50% degradation in ISR support, 
the RCP dropped again to 2.654:1 with a Blue Py of 62.4%, resulting in an additional 
1.3% decrease in Blue Py. Thus, the IRCPAT has highlighted the operational impacts 
from degradation of both external dependencies as well as ISR, which induced a 3.1% 
decrease in Blue Py. What is unique about the IRCPAT is that neither external 
dependencies nor ESINQ contributors to combat power were previously accounted for in 
the FRC or in any other operational decision support tools. Thus, the IRCPAT provides 
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users an improved operational decision support tool, supporting TSE by exploring the 
potential impacts of the OE and decisions on Blue Py. 

3. Discussion 

As highlighted in Chapter II, there are some significant gaps in current Army 
operational planning support tools, specifically in their inability to accurately account for 
contributions to RCP from non-traditional sources of combat power. This inability to 
accurately represent the OE has limited the capabilities of most tools. Eortunately, 
because the IMDP provides a method for quantifying the impacts to operational 
effectiveness from external dependencies and ESINQ effects, the outcomes of the IMPD 
can be used to improve operational decision support tools like the IRCPAT. While 
improving the ERC to address these gaps was not directly related to the intended 
contribution of this dissertation, it represents an important secondary contribution. 

The IRCPAT fills observed gaps by accounting for the contributions of external 
dependencies and ESINQ effects on the generation of combat power, providing users 
with a more detailed and complete analysis of the RCP of opposing forces. Additionally, 
the IRCPAT provides more in-depth analysis of RCP, to include assessments of Py based 
on mission set, and TSE, where the modifications of ERs and expected EERs can be 
assessed for their impacts to Blue Py. While the application described here was just a 
simple demonstration, and really only good for the specific mission set and scenario, with 
further development and execution of the IMDP to assess other mission sets, scenarios, 
and ESINQ effects, like SATCOM and PNT degradation, the IRCPAT can be improved 
even further. By improving its accuracy, usability, and relevance, the IRCPAT will be far 
more useful to current Army operational planning. These improvements will have 
immediate operational relevance to planners Army wide, and should significantly reduce 
the underestimation of combat power so routinely seen in the PRC. 

B. ACQUISITION TRADE SPACE EXPLORATION TOOL 

Similar to modeling in support of operational planning, U.S. military acquisitions 

also depends on M&S to support decision making. In fact, practitioners of MBSE place 

special emphasis on the importance of integrating M&S into the SE process. Por the most 
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part, these acquisitions support tools are software packages that engineers use to support 
systems engineering during design, development, and tradeoff analysis. Software 
packages like CORE and Innoslate provide users a host of tools to characterize the 
system through the development of key SE architecture documents. Yet these documents 
are not enough, and to implement the products in a more synchronized manner, most 
engineers employ an overarching design process to provide a more complete process for 
employing M&S. By formalizing the process for integrating architecture development 
products into external models. Beery was able to define a more accurate representation of 
the system’s interactions with the OE. Thus, the overall understanding of the system and 
its interactions are improved, yielding more insightful TSE. It is because of this improved 
capacity and the robustness of these models to explore the system trade-space through 
OEM, that the MBSE MEASA will be used for the development of the acquisitions 
support tool. The steps of Beery’s MBSE analysis methodology can be seen in Eigure 90. 
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Figure 90. MBSE Analysis Methodology. Source: Beery (2016, 86). 


The MBSE MEASA can be broken down into three phases: the systems 
architecture phase, which includes requirements analysis, functional architecture, and 
physical architecture; the modeling phase, which includes both operational and synthesis 
model definition; and the analysis phase. The execution of the MBSE MEASA results in 
linked operational and synthesis models, enabling the synchronized TSE of the system. 
When implemented in the OE captured by the operational model, the linked models 
provide a powerful tool for assessing the impacts of system design trade-offs on 
operational effectiveness. Unfortunately, even armed with improved analysis 
methodologies like Beery’s MBSE MEASA, the Army space acquisitions community has 
continued to have a fairly poor success rate with regard to its space R&D programs. This 
failure rate can often be attributed to a lack of analysis during the early phases of the 
acquisitions process, which is what MBSE analysis methodologies were intended to 
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avoid. Thus, a discrepancy was identified, highlighting some shortcomings of MBSE 
analysis methodologies that limited both their usability and effectiveness. 

The author believes the primary reason for this discrepancy is that most MBSE 
analysis methodologies use traditional MDPs to develop their models, which do not 
account for ESINQ effects. Thus, during space acquisition, which deals primarily with 
ESINQ effects, the underlying models are far more inaccurate than those used during the 
modeling of traditional (non-ESINQ) systems. These inaccurate models inform the 
decision making process and result in flawed outcomes, supporting the high failure rate 
observed in space acquisitions. The use of traditional MDPs also explains why the 
discrepancy has not been identified earlier, because traditional SE processes work 
exclusively with quantifiable data, and thus, have not generally needed to account for 
ESINQ factors. While a great improvement to previous work. Beery’s MBSE MEASA 
uses traditional MDPs, which only address explicit physical system design parameters. 
Thus, the operational and synthesis models that are so foundational to Beery’s work are 
inaccurate because they cannot account for all potential sources of combat power, 
specifically external dependencies and ESINQ effects. 

1. Application of the IMDP Outcomes 

When applied to Beery’s MBSE MEASA, the IMDP improves the underlying 
MDP, allowing the methodology to generate a more complete operational model, one 
which can account for external dependencies and ESINQ effects. The application of the 
IMDP to the MBSE MEASA can be seen in Eigure 91. 
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Figure 91. IMDP Applied to the MBSE MEASA. 

Adapted from Beery (2016, 86). 


By applying the IMDP to the MBSE MEASA we achieve four primary outcomes. 
Eirst, the IMDP supports more robust linkages between the design parameters of the 
operational and synthesis models. Previously, the MBSE MEASA used only quantifiable 
factors for these linkages, but by expanding these potential factors through the use of the 
IMDP, the linked models are now capable of accounting for ESINQ factors during TSE. 
Second, the operational model is developed using the IMDP, which provides a much 
more accurate model than the traditional MDP used by the MBSE MEASA. This 
improved understanding sets the conditions for the development of ESINQ-enabled meta¬ 
models. Third, the operational support tools can be expanded to include the external 
system of interest. While this step is similar to the MBSE MEASA, the significant 
difference is that both the tool and the system are now ESINQ-enabled, providing a more 
accurate representation of the system than possible using just the MBSE MEASA. 
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Finally, the ESINQ-enabled operational model is linked with the synthesis model using 
the MBSE MEASA. Eollowing analysis of the data from the linked models, TSE can be 
conducted to explore the impacts that design changes to the system can have on 
operational effectiveness. While this step is identical to Beery’s, the TSE conducted here 
can include considerations for ESINQ factors, allowing users to address all four of the 
potential contributors of combat power as discussed in Chapter 11. This improved 
understanding will proving a more complete exploration of the solution space during 
early phases of system design, and in turn, provide more accurate information that should 
support better decisions regarding the allocation of resources during space acquisitions. 

2. Demonstration 

To demonstrate the utility of applying the IMDP to the MBSE MEASA, a fairly 
significant amount of work was necessary to set the conditions for producing an ESINQ- 
enabled TSE tool. To avoid an overly complex and detailed demonstration of this 
application, this work will forgo as much of the detail of the MBSE MEASA as possible, 
directing the reader to Beery’s work (2016). Eikewise, rather than developing an entire 
synthesis model to support this demonstration, this work used the synthesis model 
developed by Mike Ordonez during his thesis work titled “Developing and Applying 
Synthesis Models of Emerging Space Systems” (Ordonez 2016). In his work, Ordonez 
executed a systems engineering process similar to that of Beery’s MBSE MEASA. Thus, 
by using the previous work of both Beery and Ordonez, the development of the required 
SE products and synthesis model can be avoided, and instead allow us to focus this 
demonstration on the application of the IMDP and its outcomes to improve acquisitions 
support tools. To set the conditions for this demonstration, the following mock 
acquisitions scenario and narrative was developed to drive the MBSE MEASA. 

a. Acquisitions Scenario: SmallSats 

Driven by directives from the DOD, the Army has expressed interest in acquiring 
cross-domain solutions of ISR disaggregation and redundancy. This interest is codified in 
numerous sources of doctrine, and requires U.S. forces to “change our mindset from 
simply increasing the density of ISR capabilities to evaluating our methodologies for 
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employing and integrating ISR assets” (DOD 2011b, 19). To address this requirement, 
the Army is interested in SmallSats and the utility they can bring to the ground fight. 
Unfortunately, the Army has little familiarity with space systems and has expressed 
concerns regarding their potential costs compared to their expected contributions. To 
mitigate these concerns, the Army has asked that early system analysis be conducted to 
estimate the expected impacts that SmallSats can have on operational effectiveness. 
Additionally, the Army has asked how many of these systems would be needed to 
mitigate the impact from a moderate level of ISR degradation, and what the potential cost 
of this solution would be. The Army has stated that while it would like these systems to 
be as capable as possible, it is willing to trade off goal requirements for increased access, 
its primary objective for this system, and improved resolution, its secondary objective. 
Additionally, while cost was not identified as an objective, the Army stated that it would 
be considered during analysis. The baseline Army requirements and MOEs for the initial 
analysis were provided, and are described in Table 58. 


Table 58. SmallSat Initial Requirements and MOEs 


Factor 

Unit 

Threshold 

Goal 

Improve 

Aperature Diameter 

cm 

5 

100 

MIB 

Altitude 

km 

800 

300 

LIB 

Cone Half-Angle 

deg 

60 

10 

N/A 

Mass 

kg 

200 

50 

LIB 

# of Satellites 

# 

6 

1 

N/A 

MOEs 

Factor 

Unit 

Threshold 

Goal 

Improve 

Resolution (GSR) 

cm 

300 

50 

LIB 

Total Cost 

S (Million) 

40 

10 

LIB 

Access 

min/day 

30 

60 

MIB 


These initial requirements will be used to drive the analysis of the linked models 
and will serve as the basis for the TSE conducted at the conclusion of this demonstration. 
To simplify this demonstration the following assumptions were made: threshold values 
will be treated as hard lines; the Army is not concerned with propellant; the inclination of 
the SmallSats will be fixed at 60°; and the maximum number of satellites should be 
capped at six. With an improved understanding of the scenario, the requirements, and the 
assumptions, we can now begin the demonstration. 
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b. The MBSE MEASA Step 1-3: Systems Architecture 

To highlight the potential utility of applying the IMDP to support better TSE, 
modifications to the MBSE MEASA were required to support the ESINQ-enabled 
operational and synthesis model linkages necessary for an improved TSE. Eor the MBSE 
MEASA, these modifications were fairly straight forward, with the only significant 
variation from Beery (2016) coming from two primary sources, each of which will be 
discussed in greater detail. Eirst was synthesis factor translation and mapping, which 
captured the expected impacts of the system as well as the factors that will represent 
those impacts in the operational model. Second was the establishment of input factor 
linkages, which established common inputs between the ESINQ-enabled operational 
model and the synthesis model. Additionally, while the IMDP and the EEMM can be 
used to better inform the selection of the synthesis modeling package, for brevity it will 
not be discussed here. 

(1) Synthesis Translation and Mapping (TSEl) 

The intent of this sub-step of the IMDP is to build an understanding of the system 
and link this understanding to a set of tangible effects that can be used to represent the 
system’s functions within the operational model. The IMDP leads the user to a better 
model definition of the system’s functions through a process similar to parts Ala-c of the 
IMDP. The only difference here is that we are focused on synthesis factors that are 
primarily informed by the SysME documents developed during the execution of the 
MBSE MEASA. This step focuses on the development of the synthesis half of the 
EEMM, which will be used to link operational and synthesis models, and has two 
primary purposes. The first purpose is to codify the users’ needs of the synthesis model 
by developing an operation concept. While the Use Case Diagram serves the same 
function for the system, at no time are such requirements codified for the synthesis 
model, and thus, an operational concept is useful. The second purpose is to translate the 
system functions described in the SysME products to tangible factors that will be used to 
represent those system functions within the operational model. The EEMM supports the 
mapping of synthesis input factors by providing traceability from system level functions 

to synthesis input factors; an instantiation of which can be seen in Table 59. 
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Table 59. EEMM (Synthesis Translation and Mapping) 


SynHieisInpiii 


Function SanielSysMLdocuniefltsI 


Mips 

MilFscton 

Mappine 

Tantableinpaas 

Urkinc 

Expected Impacts 

iKSbOll 

Goal 


Expected Functions ISoSImpactsI 

Mitiiatini System 

1 

/^ntSpHtl 


incraasadPcandPhofsowiMtiiticms 

i - 

Increased SMjt Pc and Pb 

300 

50 

UB 

GSP; Provide Hijb resolution imajery |cm| 


1 

Pk/Oisthjtje 


8«tr fastK dttision itialiinj and Pace of battle 

c- 

Increased collection capacity 

30 

eo 

MIB 

Access: Waxiina Meantime per day (mini 


2 

Pc 

/yf 








iSPSmallSat 

2 

teTimt/Oet 
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Sate o( Fire 











The EEMM starts by capturing the expected functions of the system, as 
characterized in the supporting SysME documentation. The system functions are then 
expanded and translated in terms of expected impacts, which are then linked to tangible 
impacts on the operational model. Einally, these effects are mapped to factors within the 
operational model that can best represent them. Although a simplified demonstration, it 
highlights the ability of the EEMM to maintaining traceability to the originating 
requirements by facilitating the user’s translation of high-level systems engineering 
functions to the operational model. With mapping complete, we can now establish model 
linkages. 

(2) Establishing Linkages (TSE2) 

To use meta-models to link the operational and synthesis models in support of 
TSE, the two models must share a common set of input factors. Recall that once 
calibration is complete, mission, agent, and external dependency factors remain fixed for 
the specific mission set, and only ESINQ factors will be manipulated in the DOE to 
account for the impacts from ESINQ effects. Thus, prior to the development of the 
synthesis model, we must ensure that these ESINQ input factors are common within both 
models. While this step is not unique, having been discussed in most works regarding 
TSE, what is original to the IMDP is the inclusion of ESINQ factors and the method by 
which these linkages are established. To link the models, the EEMM is used to map the 
ESINQ input factors from the operational model to the input factors of the synthesis 
model. The EEMM, executed for this demonstration, is shown in Table 60. 
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Table 60. EEMM (Operational and Synthesis Model Einkage) 


ESINQ Inputs 


Linked Input Factors 


Synthesis Inputs 

Ave Time/Det 

3 

-S 

Ave Time/Det 

5 


Pc 

2 

Pc 

2 

-> 

Pc 

4 


Ave Time/Det 

2 

Agent Speed 

2 

— 

Agent Speed 

3 

— 

Agent Speed 

1 


4 


Ph / D.scharee 

4 

-- 

Ph / Discharge 

4 


















Using the synthesis versus operational factor Mapping Matrix from the EEEM, 
the input factors from both models were linked. The results here show a typical outcome, 
with roughly 75% of the weighted factors being used following crosslinking. As shown in 
Table 60, RoE and Ph were both identified as operational and synthesis input surrogates. 
However, these factors were ignored during screening due to the low number of maps, 
just one each, and the fact that both factors could be accounted for in other factors, like Pc 
and Td. Additionally, while speed was included in the operational model inputs, it was 
ignored in the synthesis model and thus screened out. Thus, just two factors, Td, and Pc, 
will be used to link the models, and their output will generate the meta-models needed to 
observe the operational impacts from system design changes. 

c. MBSE MEASA Step 4: Model Definition 

Eollowing the completion of SE Architecture, the first three steps of the MBSE 
MEASA, we can move on to Step 4, Model Definition. Here both the operational and 
synthesis models are developed to meet the requirements as outlined in their respective 
OCs, using the linked input factors identified through the execution of the EEMM. 
Because the operational model was already developed, it will not be addressed here, and 
only the synthesis model development will be discussed. 

(1) Synthesis Model Development (TSE3) 

Eor this demonstration, the synthesis model developed by Mike Ordonez in his 
2016 thesis work was selected. This synthesis model was designed to analyze “the 
relationship between small satellite design inputs and outputs to provide trade space 
insights that can assist DOD space acquisition professionals in making better decisions in 
the conceptual design phase” (Ordonez 2016, v). The work of Ordonez closely parallels 
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the research interests of this work, and because of the commonality between his model 
and the operational model, no modification was needed. While the model developed by 
Ordonez was a relatively simple calculation based model, its potential applications were 
significant. His model consisted of four input/calculation tabs, two look-up tables, two 
experimental design tabs, and a trade space analysis tab, which can be seen in Figure 92. 


Trade Space Analysis 

Please submit input values for each of the blue cells below. The outputs will be 
calculated and presented in the green cells. 

Inputs Units 

GSR 

250 

cm < > 

Altitude 

200 

km < > 

Spacecraft Mass 

10 

kg < > 

Propellant 

1 

0="No", l="Yes" 

Inclination 

50 

deg < > 

tt of Satellites 

4 

< > 

Cone Half'Angle 

20 

deg < > 




Outputs 


Units 

Total Cost 

S 64,735,118.21 

USD 

Aperature Diameter 

0.87 

m 

Linear Dimension 

0.815 

m 

Number of Accesses (in 30 days) 

4.617 


Mean Access Duration 

17.755 

secs 

Payload Mass 

1.73 

kg 

Spacecraft Weighted Mass 

10 

kg 


Figure 92. Trade Space Analysis Worksheet. Source: Ordonez (2016, 79). 


The trade space analysis worksheet shown here takes seven inputs from the user, 
across a range of potential values from the threshold value to the goal value based on the 
needs of the user. The tool then provides seven outputs, which are based on meta-models 
developed following the DOE and analysis of the model. For this demonstration, these 
outputs included key estimations for resolution, cost, and access, thus providing all three 
of the primary objectives of the Army. With the models developed, a few other 
modifications were needed prior to conducting analysis and TSE. 

(2) Model Expansion for Synthesis Systems (TSE3a-3c) 

Although not required for TSE, we can capture the impact of the system in the 

reference tool now by expanding the operational model to include the system of interest. 

Because the application of the resulting synthesis meta-model to the IRCPAT is worth a 
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brief discussion, a short excursion from the TSE application of interest in this section is 
appropriate. The primary objective of this step is to better inform the IRCPAT by 
including synthesis systems of interest, similar to parts B2a-c of the IMDP. To do this, 
the IMDP helps the user better assess and visualize the expected impacts from the system 
of interest by bounding the system’s impacts through a value based process that is 
informed by both the stakeholders and the SysML products produced during execution of 
the MBSE MEASA. Eor this demonstration, simulated stakeholder interactions were used 
to develop the value curves, which were based on the authors own experience and 
expertise. The synthesis impact assessment can be seen in Table 61. 


Table 61. IMDP: TSE3a (Synthesis Impact Assessment) 


User Synthesis 
Impact 
Assessment 


Visualization 

Tools 


Linked Input 
Factor 

Modification 
for System 
Impacts 


• Import the linked input factors from part Alc/EEMM into column 1 for the 1st ESINQ factor 
of interest. 

• Identify the number of systems you want to represent, more Is better, artd wil yield a better 
exploration of the internal solution space, and drive more accurate meta-model 
development, but with the expense of resources. Create a Column for each of these. For 
this example. i consider up to 6 SmaliSats. as seen m column 2. 

• Input assessed impact change from base value) based on the number of systems for each 
of the Inked input factors into column 2. This should be informed to the largest extent 
possible by quantifiable sources of impact data, bounded by the impacts and contnbutions 
established during part B2a. Thekey is to assess as accurately as possible the expected 
maximum (Extreme) impact of the system on each linked input factor, use the value curves 
to help support the impact assessment (the curve). This should be assessed by as many $ME 
as possible, leading to an outcome that is acceptable by most. This will represent the 
improvement to blue RCP per system, things to consider mdude: 

0 Startbyconsidenngthecapabilityofthcsvstemtosupportopefations. Whatisthc 
impact of the system without any ESiNQ factors? A system can support operations in 
some areas, but not in others. 

0 IS It possible to completely mitigate the ESiNQ impacts by adding systems? Some 
impacts may be able to be completely mitigated, while others may not, regardless of 
the number of systems. 

0 This assessment aims to bound how each system reduces the impact from the ESINQ 
factors of interest on each specrfk modeling factor. 

0 This should be informed to the largest extent possible by the SysML documents, the 
EEMM, and part B2a, and if complete, the ntore thorough analysis conducted during 
TSE6. 

0 Use the value curves to help support the impact assessment, which for mitigation 
systems like used m this example, should reduce the impact from the ESiNQ factors of 
interest. 

0 This should be assessed by as many SME as possible, leading to an outcome that Is 
acceptable by most. 
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For Blue Td. l assessed a maximum impact of roughly 2M, without any diminishing 
returns as additional sateflites are added. This accounted for roughly lOM of the 
impaas from ESiNQ degradation seen in part B2a. For Pc, i assessed a 0.2S9S increase 
for each additional satellite, but with a diminishing returns factor included. This 
accounted for roughly IIH of the impacts from ESiNQ degradation seen in part B2a. 
Because this demonstration was looking at the impact of a ISR SmallSat on Blue tanks, m 
a smal protracted fight, the immediate impacts on the force will be relatively small. 
Thus, the impacts to combat effectiveness Is likely small. For other forces, larger forces, 
and longer battles, these curves could be quite different, it is for this reason that these 
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As shown in Table 61, the synthesis impact assessment is similar to step B2a and 
directs the user through a somewhat subjective, yet informed process that supports the 

estimation of system impacts on operational surrogate factors. The key difference is that 
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the majority of the assessments performed here are based on quantifiable data, primarily 
the SysML architecture products developed in the execution of the MBSE MEASA. 
Thus, this assessment is far less subjective than those conducted during ESINQ bounding. 
Eollowing this analysis, the user can then expand the operational model by including the 
impacts from the system of interest, bounded by the number of systems. Eollowing the 
expansion of the operational model to account for the synthesis system, the user can then 
execute the DOE (TSE3b), conduct analysis (TSE3c), and expand the reference tool for 
the impacts of the synthesis system. Eor simplicity, these steps will not be discussed here 
due to the fact that they are nearly identical to those from parts B2b-c of the IMDP. 
Eollowing the execution of these steps, a meta-model was developed to provide a 
mathematical representation of the expected impacts to operational effectiveness from a 
SmallSat, based on the specified mission set and task organization modeled in the 
operational model. Eollowing inclusion into the IRCPAT, the meta-model takes the 
number of SmallSats and the PR as inputs and outputs the expected impact of the system 
on RCP, which can be added to the overall RCP of the Blue force. Eor this 
demonstration, the IRCPAT discussed in Pigure 89 was updated to include a single 
SmallSat to highlight the improvement of RCP, which has been instantiated in Pigure 93. 
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Improved Relative Combat Power Assessment Tool (IRCPAT) 
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Figure 93. IRCPAT (Application Demonstration: Synthesis) 


As shown in Figure 93, following the inclusion of a single ISR SmallSat into the 
IRCPAT, the overall RCP of the Blue force was increased, resulting in a Blue Py of 
62.7%. This is an increase of 0.3% from the 62.4% seen following the inclusion of 
degraded ISR. Thus, we have highlighted the ability of the SmallSat to partially mitigate 
the impacts from degraded ISR, which in this example demonstrates that the ISR 
SmallSat is capable of mitigating roughly 23% of the 1.3% degradation to Blue RCP. 
With this short excursion to expand the IRCPAT for synthesis systems complete, let us 
move back to the TSE application of interest. 

d. MBSE ME AS A Step 5: Model Analysis 

With the operational and synthesis models developed, and the common input 
factors linked, it is now possible to exploit that linkage to provide better insight into the 
functionality of the models, specifically with regard to OEM and TSE. However, further 
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translation of the synthesis system is needed before impacts to combat effectiveness 
based on physical design changes to the system can be observed in the operational model. 
To provide more detail regarding this translation, TSE4 through TSE6 will be discussed. 

(1) A Linked Model (TSE4) 

While common input factors between the operational and synthesis models were 
established during the execution of the EEMM in parts A as well as TSEl-2, the system’s 
functional architecture has not yet been mapped to these input factors. While we know 
that the system’s impacts will be captured through the manipulation of Td and Pc, we do 
not know which system functions impact which model factors, or by how much. This step 
focuses on establishing these functional linkages as well as bounding their responses. The 
IMDP TSE4 tab demonstrates the mapping of the system level functions to operational 
model inputs. While this mapping is somewhat subjective, when bounded by the work in 
part B2a and supported by detailed SysME documentation and SME interactions, this 
process can provide extremely valuable insight. As noted by MacCalman et al. (2016), 
the goal is to map operational inputs to physical inputs either directly, or indirectly using 
“mathematical expressions, a look-up table with empirical data, or a separate type of 
model” (3). However, as Beery (2016) notes in his dissertational work, this mapping can 
also include “heuristics, and regression analysis,” which is heavily leveraged in both his 
and this work. 

As directed by the IMDP, synthesis effects mapping was accomplished for each 
of the system functions identified during the execution of the EEMM. Eor this 
demonstration, these functions included providing high resolution ISR capability 
measured as Ground Sensor Resolution (GSR) and increased visibility (access), and the 
mapping was accomplished in four basic steps. Eirst, each function was assessed by the 
user to identify the basic shape of the value curve. Then, this curve was used to estimate 
the expected response of the surrogate factor, with the expected response bounded by the 
impact results determined during part B2a. Next, a DOE was executed to build a 
representative meta-model of the synthesis design parameters and the impact of any 
change to these parameters on the primary MOE. Einally, a scaling factor was applied to 

the model to address any potential non-linearity based on the number of systems, which 
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could include synergistic outcomes as well as diminishing returns. A partial capture of 
the synthesis MOE mapping matrix for GSR can be seen in Figure 94. 


Synthesis MO€ MappingMatrix 
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Figure 94. TSE4: GSR Synthesis MOE Mapping Matrix 


The resulting meta-model from the GSR mapping matrix captured the effects of 
synthesis design changes, specifically altitude and aperture diameter, on the primary 
MOE of GSR, which are represented in the operational model through impacts to Pc. The 
mapping matrix for access was done similarly and resulted in a meta-model that captures 
the impact of synthesis design changes to altitude and cone half angle to the primary 
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MOE of access, represented in the operational model through impacts to Td. These meta¬ 
models provide the linkages necessary to construct a linked model; a description of which 
is provide by the IMDP and can be seen in Figure 95. 
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Figure 95. TSE4: Operational and Synthesis Effects Mapping 


Following the completion of the MOE mapping matrix for each MOE, the 
operational and synthesis models can now be effectively linked through meta-models. 
The four operational and the two synthesis meta-models can be consolidated, and based 
on the values of the eight primary input factors, the overall impact to Red Td and Pc can 
be calculated. With the models linked, it is now possible to execute a DOE to generate 
the data needed for TSE. 

(2) DOE (TSE5) 

Execution of the DOE here varies slightly from previous DOFs in two primary 
ways. First, high resolution designs are encouraged, where saturation of the solution 
space is imperative to developing a comprehensive TSE. Thus, these designs will be 
large, bigger than any previous design used throughout the IMDP. Second, there are 
significantly more steps required to construct the design, primarily due to the linkages of 
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the operational and synthesis meta-models. Specifically, multiple intermediate 
mathematical steps will be required to link the input factors and to finalize the design 
prior to execution. To construct the design, the IMDP directs the user through five 
general steps. First, the user identifies the primary input factors from both models. This 
demonstration included three operational inputs (#BlueTanks, BlueExtDep, and 
BlueESINQDeg) and five synthesis inputs (#SmallSats, AperDia, Alt, Mass, and 
ConeHalfAngle). In addition to these eight primary factors, two additional effects factors 
will be calculated based on the values of the eight primary input factors at each DP. 
These effects inputs included Red Td and Red Pc, and though used in the final DOE, they 
are not included in the initial design. The second step is to build the design using the 
primary input factors. To do so, the MacCalman-2ndOrderNOEH design tool was used, 
which is available from the NPS SEED Center for Data Earming at 
https://harvest.nps.edu . Using the 10 factor 2"“^ order NOEH tab, a design was built that 
had 365 DPs, with good space filling properties and little correlation between U' and 2"‘* 
order factors. This design was then stacked 10 times to increase saturation, resulting in 
3650 DPs; a partial view of which can be seen in Table 62. 


Table 62. 10 Eactor 2"‘^ Order NOEH Design Tool. 

Source: Naval Postgraduate School (2017). 
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The third step focuses on building out the remainder of the design, accounting for 
Red Td and Red Pc. To do so, the full 3650 DP design was copied to a clean spread sheet, 
and six new columns were created for each of the adjustment factor meta-models. These 
included the four operational adjustment meta-models that would represent the impacts of 

external dependencies and ESINQ degradation, as well as two synthesis adjustment meta- 
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models, which would represent the impacts from design changes to the five system 
attributes. These meta-models were applied to each DP, and their values were calculated 
based on the values of the eight primary factors at each DP. In the fourth step, two 
consolidation columns were added for Tj and Pc. Because the adjustment factors were all 
designed as modifications to the base value of either Td or Pc, the summation and 
consolidation of the six adjustment factors for each of the two effects factors was possible 
at each DP. Finally, the last step was to clean up the design by removing the 
mathematical intermediate steps used to link the effects factors. Following this 
housekeeping, the final 10 factor design was achieved, including eight primary factors 
and two effects factors, as shown in Table 63. 


Table 63. Final 2"‘^ Order NOLH Design 



Ops Model Inputs 
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For the operations model, only #BlueTanks, Td, and Pc were adjusted for each DP, 
with Blue Py being the primary output. For the synthesis model, #SmallSats, AperDia, 
Altitude, ConeHalfAngle, and Mass were used, with GSR, access, and cost being the 
primary outputs. The two unused factors were needed to establish the impacts of the 
effects factors and were needed again during TSE. Following the development of the 
design, it, the study files, and the models were submitted to the NPS Advanced 
Computing Cluster for execution, where each of the 3650 DPs was replicated 200 times, 
for a total simulation run of 730,000. 
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(3) Analysis and TSE (TSE6) 

After the models were executed, the output data from both models was 
consolidated with the original design before analysis could be conducted. This data 
included one output factor from the operations model (Py) and three output factors from 
the synthesis model (GSR, access, and cost). This was a fairly straight forward process, 
but care was taken to ensure that the output data from both models was applied to the 
correct DP in the design. Following the addition of the output data, the spreadsheet was 
cleaned up for future analysis. For IMP, this housekeeping required some translation and 
scaling of certain input factors to put them back into their original form. Additionally, the 
two effects columns (Red Td and Pc) were removed. While they were needed to execute 
the design, their impacts were now being captured by the four primary MOEs and thus, 
were removed to reduce the complexity of the analysis. A screenshot of the analysis 
spreadsheet following output consolidation can be seen in Table 64. 


Table 64. TSE Analysis Spreadsheet 


Input Factors 

Output 

Primary Factors 

Dps Outpu 

Synthesis Output 

(BlueTanki 

MueExtOet 

jeESINOOe 

rSmallSUl 

AoefOtam 

Altitude 


Mass 

Pv 

GSR 

Access 

Cost 

55 

6 

48 

6 

46.0 

497.0 

59.0 

148.0 

0.875 

1.32 

50.3 

$200,241,132 

37 

5 

67 

1 

40.0 

580.0 

13.0 

128.0 

0.26 

1.77 

0.07 

$83,146,296 

64 

47 

14 

5 

91.0 

594.0 

46.0 

100.0 

0.995 

1.23 

13.43 

$99,212,936 

39 

74 

14 

3 

15.0 

512.0 

27.0 

167.0 

0.535 

4.16 

3.39 

$143,268,680 

41 

92 

47 

4 

48.0 

663.0 

51.0 

93.0 

0.63 

1.69 

41.8 

$116,431,634 

59 

67 

100 

1 

16.0 

415.0 

44.0 

195.0 

0.955 

3.16 

2.85 

$97,199,199 

56 

73 

24 

1 

9.0 

305.0 

16.0 

196.0 

0.92 

4.13 

0.05 

$97,419,976 

56 

96 

73 

5 

61.0 

789.0 

19.0 

172.0 

0.935 

1.58 

4.33 

$200,162,265 

42 

5 

34 

1 

35.0 

429.0 

37.0 

168.0 

0.455 

1.50 

1.93 

$91,353,737 

45 

17 

29 

1 

94.0 

541.0 

37.0 

56.0 

0.635 

0.70 

3.09 

$65,255,975 

65 

93 

8 

3 

49.0 

580.0 

17.0 

190.0 

0.985 

1.44 

0.85 

$156,858,515 

83 

31 

27 

3 

84.0 

596.0 

18.0 

99.0 

1 

0.86 

1.15 

$106,134,574 


Once the data was organized, it was then imported into a IMP data table for 
analysis. Eor this demonstration, a standard least squares regression was used to conduct 
effects screening. Specifically, this regression used a factorial to the degree and 
polynomial to the degree of the eight primary factors, while using the means of all 
four outputs as the role variables. Eollowing the regression, intermediate level analysis 
was conducted to verify the usability of the output data for TSE. While this analysis will 
not be discussed here, it was done similarly to the analysis seen throughout this 
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dissertation and included investigation of solution space saturation, inspection of the 
quality of fit, ANOVA, inspection of significance, and inspection of response 
directionality and conformity. Following the intermediate analysis of the data, which 
verified the usability of the output data in this demonstration, the contour profiler was 
then used to execute TSE, which can be seen in Figure 96. 
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Figure 96. TSE6: Analysis (Contour Profiler-1) 


Using IMP, a contour profiler was set up to simultaneously observe the 
operational and synthesis modeling factors and their associated MOEs. On the left side, 
we see the operational profiler, where operational inputs can be manipulated, constraints 
on the responses established, and impacts of these decisions observed with regard to the 
linked operational and synthesis MOEs of interest. On the right side, the reciprocal is 
seen with the synthesis profiler, where synthesis input factors can be manipulated, their 
responses bounded, and the impacts of these decisions on operational and synthesis 
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MOEs observed. Using this tool, it was possible to explore the system trade space and 
build a better understanding of the system and to gain insight into how design choices can 
impact operational effectiveness. With the operational and synthesis factor settings 
selected in this demonstration, we can see that there was not a feasible solution; the 
resulting system configuration failed the meet the minimum acceptable Pv noted in the 
operational responses, as well as failed to meet the minimum GSR and access 
requirements noted in the synthesis responses. To demonstrate the utility of this tool to 
support TSE, let us explore the system trade space in an attempt to address the questions 
posed in the scenario at the beginning of this section. These questions were as follows: 
Eirst, the Army asked for an estimation of the expected impacts of a SmallSat on 
operational effectiveness. Second, the Army asked how many of these systems would be 
needed to mitigate the impacts from a moderate level of ISR degradation, and what the 
potential cost of these systems would be. Starting with the first question, the TSE tool 
was modified to remove all degradation, setting external dependencies to 100% and 
ESINQ degradation to 0%. Then, using a single satellite, the synthesis attributes were 
modified until a moderate “middle-of-the-road” solution was achieved; the TSE tool for 
which can be seen in Eigure 97. 
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Figure 97. TSE6: Analysis (Contour Profiler-2) 


This solution had a fairly low cost of $58.1 million, which was toward the lower 
(better) bound of the ranges used; had a good GSR of 1.01m, which was also toward the 
lower (better) bound of the ranges used; and provided a total access of 12.3 min/day, and 
while this was toward the lower (worse) bound of the ranges used, was still acceptable. 
While this solution was not weighted for any of the four MOEs, which is typically seen 
during system design, it provided a fairly robust solution given the lack of details for this 
question. With these system attributes, the single SmallSat was shown to provide an 
average of a 0.26% increase in Py across the range of ERs. This impact was more 
apparent at higher ERs and insignificant at lower ERs, which is understandable when 
considering the cumulative contribution of ISR to the warfighter. Thus, in addressing the 
first question, the estimated impact to operational effectiveness from a SmallSat is on 
average roughly 0.3%. In reality, the better answer is that the impact depends on both the 
design of the SmallSat, its implementation, the mission, the size of the force, and a host 
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of other factors. Nevertheless, the tool was capable of providing a quantifiable answer 
that has otherwise gone unaddressed in modern acquisitions. 

Next, the second question was considered. Here, the middle-of-the-road satellite 
design was abandoned, and the system attributes were modified to maximize the potential 
of a single SmallSat to mitigate a moderate level (50%) of ISR degradation while 
minimizing the number of satellites. Thus, Py was the primary metric used during TSE, 
and following the iteratively manipulation of the tool, a system design was identified that 
maximized the contribution to Py from a single satellite. Then, the number of satellites 
was increased until the 50% ESINQ degradation was overcome, which in this 
demonstration required four SmallSats. Einally, because there was still tradespace in the 
design, the system attributes were further refined to maximize the utility of the remaining 
MOEs while ensuring that the ESINQ degradation continued to be mitigated. This 
example focused on maximizing GSR and access, while reducing cost and complexity. 
And while this additional manipulation was not necessary to address the question, the 
step was implied when the Army stated that it wanted to maximize the attributes as much 
as possible. The TSE tool following these modifications can be seen in Eigure 98. 
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Figure 98. TSE6: Analysis (Contour Profiler-3) 


As shown in Figure 98, using these operational and synthesis attribute settings; a 
feasible design solution was obtained that met both the operational and synthesis MOE 
requirements. Thus, in addressing the second question, the resulting solution met the 
Army’s primary requirement of mitigating a 50% degradation of ISR collection assets 
through the use of four ISR SmallSats. These satellites each had a GSR of 2.51m and 
provided a total access time of 31.5 min/day, at a total cost of $111.6 million. While this 
question was again very specific, it highlights the power of the TSE tool to support 
acquisitions decision making, giving the user the capability to assess the impacts from 
design changes on operational effectiveness. 

3. Discussion 

The power of acquisitions decision support tools to effectively link operational 

and synthesis models is clearly articulated in modern works, and as described here, 

provides users an extremely flexible TSE tool for investigating system design 
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considerations and the resulting impacts to operational effectiveness. By applying the 
outcomes of the IMDP, the TSE tool was expanded, and through integration of the meta¬ 
models that could account for external dependencies as well as the impacts from ESINQ 
effects of interest, a more complete TSE tool was developed. This expansion of the tool 
gave more utility to the user, and allowed for a more accurate assessment of not only the 
operational impacts from degradation, but the potential contributions of emerging 
systems to mitigate these impacts as well. 

C. SUMMARY 

The primary contribution from the execution of the IMDP was the development of 
meta-models that describe the impacts from ESINQ effects on metrics of operational 
effectiveness. This chapter demonstrated a few potential applications of these meta¬ 
models, specifically with regard to improving operational and acquisitions decision 
support tools. Eirst, the meta-models developed during the execution of the IMDP were 
applied to an operational decision support tool. During this demonstration, the IRCPAT 
was improved to account for all four of the potential sources of combat power. This 
expansion allowed the tool to account for and quantify the contributions to RCP from 
external dependencies and ESINQ effects, which yielded a more insightful planning tool. 
Next, these improved models were used to provide a more complete and accurate 
representation of the OE during TSE, and as before, resulted in a more insightful 
exploration of the common operational and synthesis trade space. The contributions of 
this work were made possible by the ability of the IMDP to support the construction of 
better models. When these models were used as foundational elements in other decision 
support tools, the overall accuracy of these tools was improved. This improvement was 
largely due to the underlying models’ increased accuracy at representing the OE and their 
ability to account for all four sources of combat power, specifically through addressing 
the external dependencies and ESINQ effects so routinely ignored in traditional MDPs. 
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VI. CONCLUSIONS AND FUTURE WORK 


While the utility of space systems and threats from adversary counter-space 
capabilities are well understood, neither space systems nor threats can be easily 
measured; thus, they are typically ignored during the MDMP because they are considered 
non-quantifiable. Unfortunately, it is this perception of space-based capabilities and other 
external force multipliers as non-quantifiable that lies at the root of the problem. While 
ESINQ effects may be difficult to quantify, they are by no means non-quantifiable; up to 
this point, there has simply been no formal effort within the community to quantify them. 
By moving away from the consideration of external force multipliers like space-based 
effects as non-quantifiable, and rather, consider such effects as ESINQ, a more accurate 
representation of these effects can be captured in the referent. This improved referent 
addresses the possibility of quantifying the impacts of these effects while still 
highlighting the difficulty in doing so. 

The fundamental issue with traditional MDPs is that the underlying assumptions 
and methods for gathering data during the model definition steps, specifically during the 
development of the referents, are flawed. Thus, the models developed fail to represent the 
OE and the systems they were intended to model, resulting in a more incomplete and 
inaccurate analysis. To address the inability of traditional MDPs to account for the 
contributions from external dependencies and ESINQ effects, an improved and more 
flexible MDP was needed that enabled the inclusion of more than the two primary 
referent contributors. By focusing on improving the underlying MDPs to account for 
ESINQ force multipliers, it was possible to produce better models, execute more 
informed OA, and develop more complete decision support tools, which in turn should 
result in better decisions regarding preparation for operations in a D3SOE. This work 
expanded the M&S body of knowledge through the development of a formalized 
methodology to account for and bound ESINQ factors and effects within the MDP. The 
IMDP developed in this work addressed the lack of synergy in traditional MDPs and 
translated this improved understanding to a set of operational and acquisitions decision 
support tools that support the quantification of the impacts from ESINQ effects. 
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A. CONCLUSIONS 

In this dissertation, we presented a proof of concept that enabled operational 
models to account for and quantify ESINQ factors and effects. This work demonstrated 
that when applied to traditional MDPs, the IMDP improved model definition and 
development, the two primary steps of most traditional MDPs. The result of this 
improvement enabled users to gain novel insights into the workings of the model and 
account for the external dependencies and ESINQ factors and effects that had mostly 
gone unaddressed. This research clearly demonstrated the ability of the IMDP to augment 
traditional MDPs to address a broader array of potential impacts to operational 
effectiveness than previously possible. The IMDP complements traditional MDPs by 
formalizing a methodology for expanding the model definition step to account for ESINQ 
effects of interest in the referent. Through the use of the IMDP, a more accurate 
representation of the OE can be implemented in the model, greatly improving the model’s 
fidelity and ability to link a system’s characteristics — to include inputs from external 
dependencies and ESINQ effects — to metrics of operational effectiveness. The 
contribution of the IMDP to the M&S body of knowledge can best be captured by 
considering its application to traditional MDPs as seen in Eigure 99. 
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Figure 99. IMDP 


The IMDP developed in this work provides two primary advancements when used 
in conjunction with traditional MDPs: first, was the ability of part A of the IMDP to 
address the lack of synergy in traditional MDPs, specifically during the model definition 
step. By expanding the potential input sources from two to four, and then codifying the 
process for translating, normalizing, and calibrating the expected impacts that these 
contributions have on the operational model, a broader more holistic approach of defining 
a model’s referent was achieved. Second, part B of the IMDP formalized a methodology 
for implementing the improved model definition achieved during part A within the 
operational model. Through an iterative implicit modeling process, the ESINQ-enabled 
operational model was realized, and then manipulated through the modification of 
surrogate factors to capture the impacts from ESINQ effects. Not only did these 
expansions improve the utility of traditional MDPs, they resulted in a better 
understanding of the impacts of ESINQ effects on operational effectiveness, which 
supported a more holistic and complete understanding of the OE. Through the formali z ed 
definition of an IMDP, this work provided M&S users a new tool set for addressing 


267 












































ESINQ and other “soft” factors that do not fit well into traditional MDPs, addressing 
many of the M&S community gaps discussed in Chapter 11. 

In achieving this primary contribution, this work produced five secondary 
contributions that can have an immediate impact within the M&S community. These 
contributions addressed a significant portion of the organizational gaps addressed in 
Chapter II, and can better support the U.S. military as it prepares for operations in a 
D3SOE. Eirst, the IMDP formalized a MSSSM, and provided users a far more robust 
methodology for screening and selecting potential M&S packages for use in simulation 
studies. While a slight offset from the primary contribution of this work, the author found 
the lack of any formali z ed process for selecting an appropriate M&S package within the 
M&S community literature established the need for the development of the MSSSM. By 
providing users a framework for executing a more complete and logical investigation of 
available M&S packages, the likelihood of selecting a more appropriate M&S package 
for use in an M&S study has been increased. 

Second, the IMDP supported the development of an ESINQ-enabled operational 
model which can capture the impacts of ESINQ effects on operational effectiveness. 
While these effects are typically considered non-quantifiable and ignored in traditional 
MDPs, the IMDP allows model developers to address them, and thus, represent a more 
complete and realistic understanding of the OE. This expanded view of potential 
operational model inputs gives users the flexibility to explore ESINQ factors like space 
and cyber, to name only a few, and to gain insight into the potential impacts they may 
have on operations. 

Third, the IMDP, supported by DOE and statistical analysis, enabled the 
quantification of the impacts of ESINQ effects within the operational model, and thus, 
supported the development of surrogate meta-models. These meta-models can be used to 
represent the impacts of ESINQ effects in terms of impacts to operational effectiveness, 
and once developed, can be transferred and implemented within other models and tools to 
improve their accuracy. 
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Fourth, by applying the ESINQ-enabled meta-models, the primary operational 
support tool used by the Army was updated and improved. The IRCPAT developed in 
this work is a significant improvement over the current tool, the FRC, and greatly 
improves the utility and flexibility of the tool in support of operational planning. The 
IRCPAT allows the user the flexibility to address and account for external dependencies, 
ESINQ effects, and external systems when determining the overall RCP. By accounting 
for a more complete assessment of the contributions to RCP, the IRCPAT should reduce 
the underestimation of RCP that was so typical of the PRC. 

Pifth, by linking the ESINQ-enabled operational model to a synthesis model of an 
emerging system, this work was able to demonstrate the potential of developing 
improved acquisitions support tools that could better link system design decisions to 
metrics of operational effectiveness. By using an ESINQ-enabled model, acquisition 
professionals can make a more accurate assessment of an emerging mitigation 
technologies’ capacity (whose contributions can be considered ESINQ effects) to 
improve RCP. Thus, a more direct linkage between design choices and impacts to 
operational effectiveness were made, which better enforces the concept of OEM. 

This dissertation fills some significant gaps by enabling traditional MDPs to 
capture ESINQ effects. By providing a methodology that can achieve a better 
representation of the OE in the referent, model developers can bound ESINQ effects 
within a model. This more complete understanding of the OE will improve the models 
developed through traditional MDPs; will improve the assessments generated from their 
analysis; and support more informed decisions regarding both the use and allocation of 
resources. Together, these improvements will better address the gaps described in 
Chapter II, and support the United States to make more accurate and informed decisions 
as we “prepare” for operations in a D3SOE. 

B. FUTURE WORK 

This research was the logical expansion of the work of others, specifically 
MacCalman and Beery, who improved and refined their respective MBSE analysis 
methodologies to achieve a more complete and robust TSE. While this work was a proof 
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of concept, it demonstrated the capacity of the IMDP to improve the accuracy of the 
underlying models by capturing a more accurate representation of the OE, and thus, 
making it possible to improve current operational decision support tools as well as 
execute a more refined TSE. While there are countless potential areas for future work, it 
is the author’s belief that the following areas offer the most promise for expanding this 
work and continuing to improve the M&S and MBSE bodies of knowledge. 

The first area of expansion deals with the use of genetic algorithms to reduce the 
complexity and resources needed to execute the IMDP. While the author originally 
considered the use of genetic algorithms, the uncertainty at the time regarding the 
potential dimensionality issues of the solution space led to a more conservative approach. 
In this approach the author used large designs in an iterative manner to support a 
systematic investigation of the solution space, and while less efficient than genetic 
algorithms, it reduced the uncertainty regarding dimensionality. Regardless, following the 
completion of this research, the author is now confident that some efficiency can be 
gained through the use of genetic algorithms, and believes that they have the potential to 
significantly reduce the resources required to execute the IMDP. 

The second area of expansion deals with the use of the IMDP to investigate other 
ESINQ effects. While the demonstration and application of the IMDP outcomes of this 
work focused on space-based effects, there are a significant number of other ESINQ 
effects that merit detailed investigation. Such investigations could look at cyber, moral, 
leadership, and information warfare, to name just a few, and then apply that knowledge to 
expand and improve operational and acquisitions support tools. Yet, due to the broad 
distribution of ESINQ effects across numerous potential domains, it is also possible that 
the implementation of the IMDP could need adjustment. Thus, it is well worth the effort 
and resources to test the IMDP across a broader range of ESINQ effects, and if necessary, 
modify the IMPD to support the inclusion of this expanded understanding. 

The third area of expansion deals with the application of the IMDP to formalize 

the process for accounting for ESINQ factors within synthesis models. While the author 

originally had intended to apply the IMDP to both operational and synthesis model 

development, synthesis modeling was eventually considered outside the scope of this 
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work and not addressed. Nonetheless, it is the author’s belief that, like operational 
models, the application of the IMDP during the development of the synthesis model, 
specifically the model definition steps, should produce a more accurate model, yield 
better insights into the system and its interactions, and support more informed decisions. 

The fourth area of expansion deals with improving the resolution and accuracy of 
the underlying models. While this work served as a proof of concept and used relatively 
low resolution operational models to demonstrate and apply the outcomes of the IMDP, 
future expansion of this work should focus on developing a more robust model. This 
expansion should include: 

• Better source data, specifically an improvement to the COFM/FA-SWN 
used in this work, to include the use of classified data. 

• Better combat models. While MANA was sufficient for the purposes of 
this work, the author believes that, with more resources, there are far 
better models to which the IMPD can be applied. 

• Better reference tools. Like the model, a better reference tool can provide 
more resolution in the analysis following the implementation of the IMDP. 

• Different, longer, and more detailed scenarios. While the scenarios used in 
this work were fairly simple, there are significant advantages to expanding 
the size and the scope of the scenarios used in the model. 

The fifth area of expansion deals with the investigation of time, specifically the 
impact that time can have on ESINQ effects with respect to operational effectiveness. 
While this work focused on relatively short duration operational scenarios, which 
generated relatively small ESINQ impacts to operational effectiveness, the author asserts 
that the responses observed in this work are not only non-linear, but almost certainty have 
a non-linear third dimension aspect as well. While the demonstration in Chapter IV 
showed less than a 2% decrease in Py due to ISR degradation, the author believes that this 
impact would be far greater had the ISR degradation been active for a much longer period 
of time prior to the execution of combat. Thus, the author suggests that the impact of 
ESINQ effects be investigated over time, and a third dimension factor be added to the 
ESINQ-enabled meta-models to account for the change in the response that results from 
the length of time that the ESINQ effect is active. 
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The sixth and final area of potential expansion deals with the use of a more robust 
TSE tool. While this work used IMP to create a set of contour profilers that allowed the 
author to demonstrate the potential for linking ESINQ-enabled operational models to 
synthesis models to conduct a more thorough TSE, this demonstration was fairly 
simplistic and ignored a sizeable portion of the investigations typically done during 
analysis of emerging systems. Thus, this work has only provided a glimpse into the 
potential of the IMDP to improve TSE. To better articulate the advantages of using 
ESINQ-enabled operational and synthesis models during TSE, the author suggests the use 
of a more robust TSE tool, one that can provide a more holistic SE approach to systems 
and trader-off analysis. 
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APPENDIX A. MSSSM 


The purpose of this appendix is to address the first of the supporting research 
questions as outlined in Chapter 1. This question reads: What models are capable of 
representing the contributions from external dependencies and ESINQ effects, and to 
what level of resolution? By addressing this question now, in detail, using an organized 
and detailed evaluation process to investigate potential M&S packages, I will be able to 
justify my final choice for an M&S package used during this dissertation. Yet, no formal 
process for screening or the selection of M&S packages exists, so the MSSSM was 
created to fill this gap, the steps of which can be seen in Table 65. 


Table 65. MSSSM 


Part Ale (MSSSM) 


Investigate Models and identify a Model with appropriate factors which can best model the tangible 
impacts from Step B. that can be used as surrogates for the intangible effects. 


A 

Develop the M&S Operational Concept 

B 

M&S Review and Screening 

C 

Initial Screening 

D 

Secondary Screening 





1 Considerations 





2 Secondary Screening 




E 

Model Exploration (for each remaining M&S package) 


1 Initial Observations 





2 Potential Input factors for Representing ESINQ effects 





3 Potential Output responses for operational effectiveness 





4 Model development 






5 DOE 






6 Analysis 






7 Findings 





F 

M&S Comparison and Evaluation 


1 Modeling ESINQ 







2 Usability 







3 Flexibility 







4 Support 







5 DOE 







6 Analysis 







7 Agent Behavior 






G 

Conclusion and recommendation 





H 

Application (EEMM) 






1 Map the potential surrogate factors you will be using to represent the ESINQ effects you are 
investigating to the expected impacts identified during part Alb. 


2 List the final surrogate input factors you will be modifying to represent your ESINQ effects 
following your analysis. Remember, the less factors the better as long as you can represent 
all the effects and impacts you are trying to capture. 


3 Transfer the selected surrogate factors back to the EEMM 
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A. THE OPERATIONAL CONCEPT 

In most conceptual design and early phase development processes, it is often best 
to start with an overarching concept that will document how the system is expected to act, 
its intended application, how it is expected to be integrated, its expected inputs and 
outputs, and how it will be used. In SE, this document is called the OC. The development 
of a model is no different, and thus, the creation of an OC is a natural starting point. The 
creation of the OC used in this work was accomplished by addressing a number of key 
questions regarding how well a potential M&S package meets specific requirement 
considerations as outlined in the OC. These question are very similar in function to what 
Middleton (2010) calls his “Terms of Reference,” which he developed in conjunction 
with Dr. George Mastroianni in 1996 to provide users a framework for understanding the 
problem prior to the selection of a model. The MSSSM starts at this same point, but takes 
this framework further by providing an explicit methodology that takes a user completely 
through the screening and selection process. For this work, the following questions were 
used to form the body of the OC when evaluating potential M&S packages. 

1. What are we trying to investigate? 

For my research I am primarily interested in the development of an acquisitions 
decision making support tool, and secondarily, the creation of an operational support tool 
in order to provide better information to support better operational and resourcing 
decisions. These tools will tie synthesis and operational design tradeoffs of emerging 
space systems with impacts to combat effectiveness. Thus, I need a combat model that is 
capable of providing data capable of quantifying decision impacts to measures of combat 
effectiveness within the context of a D3SOE. 

2. What types of studies are you interested in (FiveWirtual/Constructive)? 

Simulations can be broadly broken into three primary groups: Five, where real 

people use real equipment; Virtual, where real people use simulated equipment; and 
Constructive, where simulated people use simulated equipment. Because my tools will 
require the use of meta-models in order to more accurately account for uncertainty, I will 
require a significant number of simulation runs to fully explore the tradespace. Coupled 
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with the fact that I have limited time and resources available to me as a student and 
researcher, I will only be considering constructive simulations. While Live and Virtual 
M&S tools have potential merits with respect to my work, they are heavily dependent on 
levels of human integration and time that is beyond my capacity to provide. 

3. What is the purpose of the study (Descriptive/Prescriptive/Predictive)? 

This will be a mixed methods study, where I will be interested in all three study 

purposes. I am interested in the descriptive nature of the model to allow me to gain novel 
insight and quantify the potential impacts from a D3SOE and potential mitigation 
strategies, which will support operational planning and support. I am interested in the 
prescriptive nature of the model to allow for comparisons between competing technology 
alternatives, which will support acquisition decision support. And finally, I am interested 
in the predictive nature of the model to allow for the anticipation of operational impacts 
from adversary counter-space activities, which will support operations and planning. 

4. What are the desired factors, responses, MOPs, MOEs? 

A factor is simply a user’s input into a model, or the settings that establish the 
initial conditions of the simulation. The responses on the other hand are the outputs from 
the M&S following its completion. Eooking at this from the perspective of OEM, my 
primary evaluation metrics or MOEs will be in the form of measures of combat 
effectiveness. Thus, I will need an M&S package capable of accepting combat related 
input factors and noise factors, and then output combat related responses from the 
perspective of ground forces. Any M&S package that takes in user factors related to 
combat, attrition rates, probabilities of kill, hit, detections, etc., and then outputs 
responses like casualties, length of battle, communication effectiveness, shooter-targets 
stats, ect, is preferred over M&S packages that have less intuitive responses. 

5. What level of the model hierarchy do you want? 

The purpose of this work is to try to evaluate emerging space systems and their 
capabilities to mitigate impacts to combat effectiveness due to a D3SOE. I am attempting 
to do this all within the context of the OE described in the recently published AOC. Thus, 
I am looking for a combat model that can accurately model ground combat operations at 
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the Battalion to Brigade level, which is reliant on reach back support from 
communications and space. Therefore, a mission level model is most appropriate for this 
dissertation. While one could argue that a Theater level model is also valid, these models 
tend to be more deterministic, less detailed, and more complex than what I need for this 
work. 

6. Do you need a deterministic or a stochastic model? 

As always this should be based on the intended needs, wants, interests, and 
requirements of the stakeholders. A deterministic model is one where variability and 
uncertainty are ignored. These are typically simpler, good for point estimates, and more 
controllable. Stochastic models on the other hand take in account uncertainty and 
variability, where a given set of inputs will produce a range of outputs along some 
distribution to account for randomness. Because I am looking for a combat model that 
can accurately quantify combat and the impacts to combat effectiveness, coupled with the 
fact that combat by nature is inherently chaotic, a stochastic model is appropriate. 

7. What level of resolution (Low to High) is needed? 

Resolution is simply the degree of detail or fidelity of the model. The higher the 
resolution the more accurately it represent the real world, but at the cost of increased 
complexity and resource requirements. Because this is a proof of concept, and time and 
resources are limited, I will be looking for a relatively low resolution model. While not as 
accurate in its representation it will meet the intent of this research and can be expanded 
upon in the future with I higher fidelity model. 

8. What are the requirements for VV&A? 

VV&A establishes the fitness and credibility of an M&S for a specific purpose 
and use. Because I am not developing an M&S, verification is not considered, but 
Validation and Accreditation should be addressed because my VV&A requirements will 
impact the number of potential M&S packages. Because this is a proof of concept, with 
the intent to develop a method and set of tools for supporting operational and acquisitions 
decisions making, I do not need the model to be VV&A. While it would be beneficial, it 
is not required at this juncture, and as long as the selected M&S meets face validation it 
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will meet my intent for this research. In the future the methodology and tool set provided 
by this work can be expanded to incorporate a VV&A model. 

9. Do you need a Time-Step or a Discrete Event model? 

Both Time-Step and Discrete Event (DE) are methods of time advancement 
within a simulation. Time-Step models advance the simulation clock by a fixed 
increment, at which time the states of all agents are updated. These are more intuitive, but 
can be computationally excessive and they induce anomalies and artifacts due to the size 
of the time step. DE models advance time to the time of the next event in the event list, at 
which time the states of just that event are updated. These are far more efficient, 
requiring far less computational power, but complexity increases with the number of 
agents and interactions. Because neither of these impact the ability of an M&S package to 
meet my needs both are valid options at this point, so I will ignore it as a screening factor. 

10. What type of output and results are needed? 

Again, I am interested in the output of the simulation, specifically how a given set 
of input factors affect the output response in terms of ground combat operations. But, 
because I mentioned that I will need to perform face validation, I will also be interested 
in the behaviors of the M&S as the simulation progresses. So, I will need an M&S 
package that is capable of producing outputs in terms of measures of combat 
effectiveness that also gives me visibility inside of the simulation as it unfolds. This 
typically means a time-step model, but there are DE models available that allow users to 
visualize the model as it unfolds. 

11. What level of classification is needed? 

To simplify the complexity of the problem, and for ease of research, publication, 
and model selection, I will be doing this dissertation at the UNCEASSIEIED level. While 
this will likely not accurately quantify the number of adversary threat capabilities and 
friendly mitigation capabilities, there is enough information available at the unclassified 
level to accurately capture the full range of potential effects needed to complete the proof 
of concept, and develop an acceptable methodology and tool set. Euture expansions of 
this work should use higher levels of classifications in order to more accurately (reduced 
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variability) represent the impaets to operations from adversary use of counter-spaee 
eapabilities. 

B. M&S REVIEW AND SCREENING 

In order to develop a set of potential M&S paekages, I started my assessment by 
doing a broad investigation of all M&S paekages that were eurrently available, either 
through NFS, serviee M&S organizations, or the DOD M&S Catalog. I defined the term 
“available” as any M&S paekage that Army planners eould get access to and utilize for 
planning and experimentation, without cost. At this early juncture, I was more inclusive 
than exclusive and included any and all M&S packages that looked as if they could be 
modified for my purposes as outlined in the OC, without constraints. My investigation 
uncovered the M&S packages described in Table 66 that at first glance looked as if they 
could meet my needs. 


Table 66. Potential M&S Packages 



OoD M&S Catalog 

NPS 

AF Space 

Army Training 

AF 

Army 

MANA JDAFS OSM 

SEAS sa SB SIAM Guardian 

JCATS AWARS JTIS OneSAF 

AFSIM 

COMBATXXI 


As shown in Table 66, my initial investigation uncovered fourteen potential M&S 
packages, which are roughly organized by source and then by purpose. The majority of 
the packages came from the DOD M&S Catalog, with AF Space and Army Training 
models accounting for the majority of them. I looked at each of the fourteen M&S 
packages in a little more depth to explore the potential of each to meet my research 
needs, and a brief synopsis of each potential M&S package follows. 

1. MANA (Map Aware Non-Uniform Automata) 

MANA is an agent-based model that was developed to conduct military OA, 
specifically focusing of CO A and tradeoff analysis. MANA was developed around two 
key ideas: First, “that the behaviour of the entities within a combat model (both friend 
and foe) is a critical component of the analysis of the possible outcomes. [Second], that 
we are wasting our time with highly detailed physics-based models for determining force 
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mixes and combat effectiveness” (McIntosh et al. 2007, 2). MANA is of interest because 
of its focus on the impacts of communications, sensors, and SA on combat effectiveness. 

2. JDAFS (Joint Dynamic Allocation of Fires and Sensors) 

JDAFS is a low resolution DE modeling framework that provides users a nearly 
unlimited amount of modeling freedom. JDAFS was developed to support users in 
“situations requiring fast turnaround analysis and those requiring much flexibility and 
customization on the part of the model” (Buss and Ahner 2006, 4). Because it is a 
framework, it is possible for users to add in almost any functionality required, but at a 
cost of development time and complexity. 

3. NPS-OSM (NPS-Orchestrated Simulation through Modeling) 

OSM is a DE simulation model currently in development by the SEED Center at 
the NPS using the Orchestrated Simulation through Modeling (OSM) modeling and 
simulation framework. The focus of NPS-OSM has been on modeling maritime 
maneuver and combat, but currently the model is still fairly immature, with no ground 
combat capability and no explicit communication or BMC2 functionality. 

4. SEAS (System Effectiveness Analysis Simulation) 

SEAS is a constructive agent-based combat model framework that focuses on Air 
Eorce Air and Space operations. SEAS was developed to support acquisitions decisions 
of emerging capabilities early in the development life cycle, and is used to support the 
“exploratory analysis of new system concepts, system architectures, and Concepts of 
Operations (CONOPS) in the context of joint warfighting scenarios” (TeamSEAS 2017). 
SEAS seems to have some significant capabilities regarding simulating space, space 
capabilities, and space support activities to warfighters in a D3SOE, which is of direct 
interest to my work. 

5. SCT (Space Capability Tool) 

SCT is a model prototype that consolidates the functionality of many other Air 
Eorce models. It provides Air Eorce users with a “Google Earth-based tool that provides 
the impacts to Space and the Warfighter, as well as mitigating Tactics, Techniques and 
Procedures (TTPs) if Space capabilities are degraded or destroyed” (DOD M&S Catalog 
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2017f). It does not model ground combat operations well, but because of its classified 
nature, is capable of taking national level intelligence and capabilities to more accurately 
assess space impacts to Air Force operations. 

6. SB (Space Brawler) 

SB is a data package designed to supplement SEAS by providing the space 
community with a government owned “baseline, version control model of space 
operations that provides a standard point of departure, which can rapidly be modified or 
tailored for quick turn studies with a focus on space” (DOD M&S Catalog 2017e). SB 
significantly reduces the resource and time requirements necessary to develop a SEAS 
scenario by providing an operational foundation of steady-state space operations. 

7. SIAM (Space and Information Analysis Model) 

SIAM is an effects-based targeting support tool used to optimize collection and 
targeting priority of available weapon systems against adversary systems. SIAM can be 
used to “display communications paths, identifies choke points, prioritizes targets, 
assesses weapons planning, and identifies intelligence collection shortfalls” (DOD M&S 
Catalog 2017g). This is a useful tool for addressing the impacts from adversary counter¬ 
space threats and prioritizing them for targeting. 

8. Guardian (GUARDIAN) 

Guardian is a visualization tool used to assess when a specific space system may 
be vulnerable to adversary counter-space activities. It provides users a PC-based 
vulnerability analysis tool for the “visualization and analysis of space system 
susceptibility to counterspace threats” (DOD M&S Catalog 2017c), but it does not assess 
the impacts from these possible attacks. This is a planning tool, which is useful for 
anticipating periods when systems can be attacked based on threat capabilities. 

9. JCATS (Joint Conflict and Tactical Simulation) 

JCATS is an interactive (Virtual) M&S tool that is used by the Army to support 
training, analysis, and planning. JCATS is capable of simulating a wide range of OEs, as 
well as a wide range of mission sets and force structures. Because of its primary use as a 
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training tool, JCATS requires a significant amount of human input and resources to 
develop and execute scenarios. 

10. AWARS (Advanced Warfighting Simulation) 

AWARS is an Army comparative analysis tool designed to provide users a multi¬ 
sided, deterministic, DE simulation environment that can represent “land and amphibious 
warfare from brigade combat team (BCT) to division or JTF level” (DOD M&S Catalog 
2017a). The Army uses AWARS for a wide range of activities to include concept 
exploration, experimentation, force structure analysis, as well as trade-space comparison. 

11. JTLS (Joint Theater Level Simulation) 

JTLS was designed to support the operational planning, training, and analysis 
requirements of joint planning staffs, specifically at higher level organizations like BDE 
and above. JTLS is an “interactive, multi-sided wargaming system that models a joint and 
coalition force air, land, and naval warfare environment” (DOD M&S Catalog 2017d), 
and thus, has little relevance to tactical and operational level organizations. 

12. OneSAF (One Semi-Automated Forces) 

OneSAF is a variable resolution M&S package originally designed to meet the 
training requirements faced by the Army during its recent transformation. OneSAF is an 
entity level model, specifically focused on supporting “constructive and virtual training, 
computer-generated forces, and mission rehearsal designed for brigade-and-below, 
combat, and non-combat operations” (U.S. Army 2013b, 264). While OneSAF was 
developed to supports the wide range of M&S tasks, to include constructive modeling, it 
was primarily designed to support virtual (human in the loop) training. 

13. AFSIM (Adv. Framework for Simulation, Integration and Modeling) 

AFSIM is an Air Force agent-based model designed to simulate the full range of 
joint operations from ground to space, at various levels of resolution, at the mission and 
below level. AFSIM consists of three components, which include a framework, an 
Integrated Development Environment (IDE), and a visualization tool, and provide users 
with “a flexible and easy to use agent modeling architecture which utilizes behavior trees 
and hierarchical tasking” (Clive et al. 2015, 73). AFSIM is a fairly complex model, and 
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although it has primarily been used to model spaee and air operations, should be eapable 
of modeling ground operations in enough detail to make it useful. While it is ITARS 
restrieted, and requires approval for use, it is available through request. 

14. COMBATXXI (Combined Arms Analysis Tool for the 21st Century) 

COMBATXXI is the premier Army eombat model, and was developed to replaee 
past M&S paekages in order to meet eurrent and future Army M&S needs. The goal was 
to design a more aeeurate high resolution entity level “analytical simulation tool used for 
weapon systems and tactics evaluation in brigade and below combined arms conflicts” 
(DOD M&S Catalog 2017b). By tying modeling responses to metrics of combat 
effectiveness, COMBATXXI greatly aids operational support to a high level of detail. 

As it stands, a list of fourteen potential M&S packages was a fairly large group to 
investigate in any detail, so I thought it was best to start by screening out any M&S 
packages that would not meet my initial, hard line research needs. To do this effectively, 
I started by formally defining and documenting what “my needs” were in order develop a 
better understanding of the problem. By carefully outlining how I wanted the model to 
act, and what the intended purpose is, I was possible to significantly screen potential 
modeling platforms to a more manageable list for explorations. 

C. INITIAL SCREENING 

With the initial research complete, a matrix was created to examine how each of 
the fourteen potential M&S packages addressed the requirements as outlined in the OC. 
Some of these requirements were non-quantifiable, but based on my subjective 
assessment of each M&S package’s ability to meet the requirements I attempted to be as 
inclusive as possible. Thus, I only screened out M&S packages that I was fairly certain 
could not meet my needs as outlined in the OC. The initial Screening Matrix can be seen 
in Table 67. 
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Table 67. Initial Screening Matrix 



As shown in Table 67, of the fourteen potential M&S packages examined, seven 
(highlighted in red) failed to meet the baseline requirements for my research as outlined 
in the OC, and thus, would be excluded from further consideration. The most common 
reason for exclusion was the inability to adequately model ground combat operations. 
With my research focusing on capturing impacts to combat effectiveness of ground force 
in a D3SOE, I must have a combat model capable of adequately representing ground 
combat, to include appropriate response factors and MOEs. This screened out three Air 
Eorce M&S packages and one Navy M&S packages (OSM, SCT, SIAM, and Guardian). 

Two more Army training M&S packages were also screened out because they 
were not constructive (JCATS and JTES). While human-in-the-loop simulations are 
powerful training and analysis tools, they do not lend themselves to DOE where 
thousands of simulation runs would be needed to effectively evaluate COAs and 
competing design alternatives. 

Einally, another Army training M&S package was screened for being 
deterministic (AWARS). While deterministic M&S packages have their place, I do not 
find deterministic models appropriate within the context of my research where the 
variability of combat will be intrinsically important to capture the effects of a D3SOE on 
combat effectiveness. A more detailed description of why each of these seven potential 
M&S packages was screened follows. 

1. OSM 

OSM is in early development, and is not capable of meeting many of my needs as 
outline in the OC without significant application of resources, namely in the complete 
development and integration of the ground component and communications architecture. 
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2 . 


SCT 


SCT is a Web-Based JWICS M&S package prototype, and while it has potential 
for follow on work in classified expansions to my work, its status as prototype and its 
classification rule it out for potential use. Additionally, because it is an Air Force model, 
the complete development of the ground component would be required. 

3. SIAM 

SIAM is a targeting tool used for prioritization, allocation, and assessment of 
weapons effects on targets. It is an optimization tool, not a simulation in the sense that 
behaviors can be monitored. There are no means in which to include ground combat 
forces into this model, and thus it fails to meet a primary requirement for my needs. 

4. Guardian 

Guardian is a vulnerability assessment tool for assessing space system 
susceptibility to known counter-space threats. It is a tool, not a simulation, and while it 
can be used for analysis, it has little use in tying space vulnerabilities and threats to 
impacts on combat operations, and no capability to model ground combat operations. 

5. JCATS 

JCATS is an interactive (non-constructive) training, analysis, and mission 
planning/rehearsal tool. It was designed to train large groups of people, not for use in 
large DOE experimentation. It requires extensive time and resources for employment, 
with numerous humans in the loop simulations running for days at a time. Thus, there is 
no way to generate the required amount of data needed to be useful in my work. 

6. AWARS 

Is a deterministic combat model with significant operational and performance 
input requirements used for studies and analysis. It is more geared toward training and 
planning than simulation, and its nature as a deterministic model make it unusable for my 
purposes, which rely heavily on modeling uncertainty in order to capture variability. 
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7. 


JCATS 


Like JTLS, JCATS is an interactive (non-constructive) analysis tool for use in the 
development of OPLANS, and thus, for the same reasons as JCATS, is not suited for my 
purposes. After removing these seven M&S packages based on failure to meet 
requirements as outlined in my OC, seven potential M&S packages remained, which can 
be seen in Table 68. 


Table 68. Remaining M&S Packages after L‘ Screening 



DoD M&S Catalog 

Screening Matrix 

NPS 

AF Space 

Training 

AF 

Army 

Factors 

MANA 

JDAFS 

SEAS 

SB 

One SAF 

AFSIM 

OMBATXX 

Unclassified 

Yes 

Yes 

Yes 

ITAR 

Yes 

ITAR 

Yes 

Access/Availability of M&S 

Yes 

Yes 

Request 

Request 

Request 

Request 

Yes 

Ground Force Opperations 

Yes 

Yes 

Limited 

Limited 

Yes 

Limited 

Yes 

Space Enabled 

Limited 

Limited 

Yes 

Yes 

Limited 

Yes 

Limited 

Stochastic Model 

Yes 

Yes 

Yes 

Yes 

Yes 

UNK 

Yes 

Iponstructive 

Yes 

Yes 

Yes 

Yes 

Both 

Yes 

Yes 


D. SECONDARY SCREENING 

Once I had screened the list of potential M&S platforms based on the questions 
asked in developing the OC, I was then able to do additional screening based on other 
additional, “secondary” factors that while not critical to the success of an M&S Packages 
ability to meet my primary needs, can have an impact on my ability to efficiently conduct 
my research. These secondary screening considerations focus on aspects like usability, 
ease of use, and resource requirements to name just a few. 

While individually these will likely not screen any M&S packages from 
consideration, if any M&S package fails to meet a significant number of these secondary 
considerations it is justifiable to screen them from further consideration. While this is a 
subjective assessment, due to the fact that I am the one conducting the research I believe 
that it is acceptable. With this in mind, and because the purpose of this dissertation is to 
provide a proof of concept and to develop and describe the methods and tools required to 
meet my research objectives, I was able to highlight some secondary screening 
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considerations with regards to any potential M&S packages by addressing the following 
usability considerations. 

1. Are there existing models? 

It is advantageous to use an existing model and modify it to my purposes rather 
than to develop my own from scratch. Starting from an established model will greatly 
reduce the amount of effort and time required to develop the highly detailed model needed 
for my research. 

2. Is there access to the M&S package and additional resources? 

It is advantageous to use an M&S package that is freely available, with a detailed 
manual and resident expertise and support here at NPS. Using an M&S package that is 
highly used at NPS would increase the availability of resources and support which could 
greatly reduce the time and effort of model development, data production, and analysis. 

3. How usable is the M&S package? 

It is advantageous to use an M&S package that is easy to use, has a quick learning 
curve, is capable of rapid development and iterative modification, as well as having a 
relatively low level of complexity. Because my work would require significant 
development and T&E to reach face validation of the operational model, an M&S package 
that could be rapidly modified and run repetitively without the use of external IDEs or 
packages was advantageous because it would greatly decrease the complexity of the overall 
process. 

4. Can it model ESINQ effects? 

It is advantageous to use an M&S package that has the capability to model 
directly or through indirect representation the impacts of a D3SOE on combat 
effectiveness. Having an M&S package that already had the “knobs” to turn to represent 
the effects of counter-space activities would be greatly reduce the development time 
needed to represent these effects in the combat model. 
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5. 


Does it support large scale DOE at NFS? 


It is advantageous to use an M&S package that is capable of being executed 
following a specified DOE on the NFS computer supper cluster. Any M&S package that 
has already had execution scripts developed for use with the cluster would be of further 
use. This would allow for the execution of a large number of simulation runs without a 
significant amount of resources to build the required scripting language to execute the 
M&S package according to a DOE on the computer cluster, which will drastically 
decrease the time required to produce output data, and allow for more routine use of the 
cluster which will increase the development tempo. 

Taking these five considerations into account and applying them to the initial 
screening matrix seen in Table 68, it was now possible to conduct a secondary screening 
of the remaining potential M&S packages based on the additional considerations in order 
to further reduce the number of potential M&S packages. Table 69 shows the status of the 
remaining candidate M&S packages following this secondary screening. 


Table 69. Fotential M&S Fackages Following Secondary Screening 



DoD M&S Catalog 

Screening Matrix 

NPS 

AF Space 

Training 

AF 

Army 

Factors 

MANA 

JDAFS 

SEAS 

SB 

One SAF 

AFSIM 

OMBATXX 

Unclassified 

Yes 

Yes 

Yes 

ITAR 

Yes 

ITAR 

Yes 

Access/Availability of M&S 

Yes 

Yes 

Request 

Request 

Request 

Request 

Yes 

Ground Force Opperations 

Yes 

Yes 

Limited 

Limited 

Yes 

Limited 

Yes 

Space Enabled 

Limited 

Limited 

Yes 

Yes 

Limited 

Yes 

Limited 

Stochastic Model 

Yes 

Yes 

Yes 

Yes 

Yes 

UNK 

Yes 

Constructive 

Yes 

Yes 

Yes 

Yes 

Both 

Yes 

Yes 

Level of Resolution 

Low 

Low 

Variable 

Variable 

Variable 

High 

High 

Complexity 

Low 

Mod 

High 

Mod 

High 

High 

High 

Updated Model (current) 

Yes 

Dev 

Yes 

Yes 

Yes 

Yes 

Yes 

Mission level 

Mission 

Mission 

Variable 

Variable 

Variable 

neaemen 

Mission 


As shown in Table 69, of the seven remaining M&S packages, four (highlighted 
in Dark Orange) were identified for screening due to a combination of factors that 
contributed to a high level of uncertainty regarding the amount of effort needed to use 
them in support of my work. Three failed to meet the secondary considerations for this 
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research, which screened out one Air Force and two Army M&S packages (AFSIM, 
OneSAF, and COMBATXXI). Thus, these would be excluded from further consideration 
as potential M&S packages. The primary reason for exclusion was the relatively high 
resolution and complexity of the models. With my research focusing on providing a proof 
of concept, an overly high resolution model would likely induce unneeded complexity. 
While having a combat model capable of adequately representing ground combat is 
needed, it does not need be overly complex or detailed as long as the effects are 
accurately represented. Thus, high resolution models, while more accurate, are not 
necessarily needed for this work. If my work is successful, future expansion of this work 
can include the use of higher resolution and higher complexity models. A more detailed 
description of why each was screened out follows. 

1. AFSIM 

AFSIM is currently used within some Air Force communities as an Air combat 
model, and while it is a fairly powerful M&S package, it does not treat ground combat 
equally. Thus, there would be a significant learning curve and long development time to 
implement any ground combat scenario, and the accuracy and functionality of this model 
would be questionable. AFSIM is an detailed and highly complex space model that 
utilizes a fairly low resolution representation of ground combat. The combination of these 
factors induces too much uncertainty to seriously consider it as a potential M&S package 
for my research. 

2. OneSAF 

While OneSAF is heavily used by the Army, it was primarily developed as a tool 
to support virtual training. While it has capabilities to support constructive modeling, it 
was not built with that as the primary purpose. Thus, use as a constructive tool still 
requires a significant amount of manpower and resources compared to other purely 
constructive M&S tools. Thus, it would not be considered further. 

3. COMBATXXI 

COMBATXXI is likely the best combat model which I investigated, and easily 
has the most capability of any of the M&S packages I investigated to model ground 
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combat, with the potential for modeling a D3SOE. Unfortunately, it was by far the most 
complex of all of the models I investigated, with a high level of detail. While I will 
recommend the Army use it during the future expansion of this work, its high level of 
complexity and detail are simply overkill with regard to this dissertation. 

This left four potential M&S Packages as seen in Table 69. While these four met 
both my initial and secondary screening requirements, one key issue needed to be 
addressed before moving forward. This issue revolved around the fact that SB is not an 
M&S package, but rather a data package for implementation within SEAS. This package 
provides a complete simulation foundation of steady state operations for SEAS model 
developers to start from in order to significantly reduce the complexity of the 
development phase. Thus, some additional screening consideration was needed. 

One final screening consolidation was done to take advantage of the linkages 
between SEAS and SB. SEAS is a simulation framework, and by itself, would require a 
significant development effort to achieve a usable model for my dissertation. While a 
powerful tool that is likely fully capable of meeting my needs, when considering the 
limited resources available to me, the feasible of using SEAS as my M&S package comes 
into question. Enter Space Brawler. Thus, if SEAS is used with SB, it would likely 
provide enough of a foundational starting point to make the use of the SEAS package 
feasible. Therefore, the combination of the two will be included it in the detailed 
screening. Table 70 shows the remaining three M&S packages that I will be considering 
in the detailed exploration. 
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Table 70. Final M&S Packages 



DoD M&S 

Screening Matrix 

NPS 

AF Space 

Factors 

MANA 

JDAFS 

SEAS/SB 

Unclassified 

Yes 

Yes 

ITAR 

Access/Availability of M&S 

Yes 

Yes 

Request 

Ground Force Opperations 

Yes 

Yes 

Limited 

Space Enabled 

Limited 

Limited 

Yes 

Stochastic Model 

Yes 

Yes 

Yes 

Constructive 

Yes 

Yes 

Yes 

Level of Resolution 

Low 

Low 

Variable 

Complexity 

Low 

Mod 

Mod 

Updated Model (current) 

Yes 

Dev 

Yes 

Mission level 

Mission 

Mission 

Variable 


As shown in Table 70, I have selected three potential M&S packages for further 
in depth investigation for utility as potential tools for use in my dissertation. While the 
selection of these potential M&S packages may draw some scrutiny, understand that the 
purpose of this dissertation is to define a methodology and develop decision support tools 
for the operational planning and acquisitions resourcing of emerging space systems 
through effectiveness based decision making. Thus, the method and tools, while not as 
accurate as can be, will meet the intent of this dissertation. If successful, it is my hope 
that the Army Space Operations, R&D, and Acquisitions communities sees the value that 
these methods and tools have, and will invest resources to expand the depth and breadth 
of my work, and creating a more robust methodology and tool set. This could include the 
use of better and more detailed models (COMBATXXI), VV&A M&S packages, 
classified information, higher levels of resolution, longer/larger combat scenarios, as well 
as using numerous different combat scenarios to produce a more robust outcome. 

The following sections will explore in more detail the three selected M&S 
packages identified in Table 70. Each section will be devoted to a single M&S Package, 
and will cover: my initial observations; how the M&S package accounts for 
communication, GPS, and ISR degradation; potential input factors for representing a 
D3SOE; potential responses for combat effectiveness; model development; data output; 
DOE; analysis; and finally, my overall findings. The intend of this investigation is to 
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gather a much more detailed understanding of each M&S package’s capabilities to 
support my research, and once compete, to weight each against each other in order to 
select the best M&S package for my use. 

E. MODEL EXPLORATION 

1. Exploration of MAN A 

MANA was one of the first combat models I was exposed to, and there is a wealth 
of resident experience and knowledge at NPS that greatly aided in my investigation. 
MANA was designed to allow users to explore a wide range of scenarios at a relatively 
low resolution, and was developed around two key ideas: First, “that the behaviour of the 
entities within a combat model (both friend and foe) is a critical component of the 
analysis of the possible outcomes. [Second], that we are wasting our time with highly 
detailed physics-based models for determining force mixes and combat effectiveness” 
(McIntosh et al. 2007, 2). Having interest in modeling combat, especially how agent 
personalities can impact combat effectiveness, this statement peaked my curiosity. When 
combined with the fact that one of MANAs goals is to support the modeling of 
communications and the sharing of SA, which is essential for implementing and 
degrading space dependencies, led me to conclude that MANA may be a appropriate 
M&S tool for modeling combat operations in a D3SOE. 

a. Initial Observations 

My initial investigation of MANA was focused on getting a feel for the usability 
of the model, how it operated, and how easily it could be developed and modified. 
Additionally, I wanted to investigate the factors within the model that could be modified 
to “represent” the potential impacts from operations in a D3SOE. When it comes to 
usability, MANA is extremely user friendly, with a relatively quick learning curve. It 
comes with a pretty detailed user manual, and there are numerous experts available 
locally to support model development and troubleshooting. MANA has been extensively 
used in research at NPS, especially in the OR department where it has been used as the 
primary M&S package for more than twenty thesis and at least three dissertations. Thus, 

there are ample model available from which I can use to build my work. 

291 



My initial investigation included a look at the factors in MANA that I thought I 
could use to model the impacts from a D3SOE. I did this by manipulating various factors 
individually to develop a better understanding of how the model used these factors. What 
I discovered, as I expected, is that there is no direct/specific factor settings for degrading 
communications, ISR, and GPS. Luckily, there seem to be ample ways to “represent” the 
expected impacts and effects of a D3SOE through the modification of other existing 
factors within MANA and using these factors as “surrogates” for the expected impact. 
The following MANA input factors were identified as potential surrogate factors for 
which modification could represent impacts from counter-space activities in a D3SOE. 

b. Potential Input Factors for Representing a D3SOE 

A major focus of my work is in the modeling of combat operations within a 
D3SOE. Broadly speaking, the effects of a D3SOE on combat operations can be binned 
into three distinct groups. The first is communications degradation, when an adversary’s 
use of counter-space capabilities restricts the flow of friendly information. Second is ISR 
degradation, when an adversary’s use of counter-space capabilities restricts a friendly 
forces’ ability to collect information. The third is PNT degradation, when an adversary’s 
use of counter-space capabilities degrades the accuracy and availability of critical 
position and timing signals of friendly forces. During my brief investigation of MANA I 
inspected agent attributes and factor settings that I believed could potentially be modify 
and used as a method for representing the effects of a D3SOE with respect to 
communications, ISR, and PNT degradation. 

Communications Degradation 

Here, from the development of MANA, there are several potential factors which 
we can use to degrade communications. Since each agent shares information with others 
through communications links, it is easy to build accurate communications links to 
represent real world communications. These links could then later be degraded to various 
levels based on the expected impacts from adversary jamming. Eactors which I identified 
as being likely surrogates for representing these effects were: Inter-Squad Comms 
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Accuracy, Inter-Squad Comms Latency, Inter-Squad Comms Reliability, Inter-Squad 
Comms Capacity, and Intra-Squad Comms Delay, which I will now define. 

• Latency 

The MANA manual describes latency as the “number of time steps taken for each 
message to reach the receiving squad” (McIntosh et al. 2007, 70). Latency was a fairly 
significant factor and seems to be a good factor for representing impacts to 
communications and SA. Care will need to be taken when using latency to accurately 
depict communications and the passing of SA because MANA assumes instantaneous 
dissemination of SA throughout the squad after it is successfully communicated. Thus, 
delays in incorporation of the SA throughout the squad will need to be added to the 
overall latency if desired. 

• Reliability 

The MANA manual describes reliability as “the likelihood that a given message 
will be successfully sent on the link per attempt” (McIntosh et al. 2007, 70). Reliability 
was also a somewhat significant factor and seems to be a moderately useful factor for 
partially representing the impacts to communications and SA error. While its overall 
impact was minimal, it was enough to keep for further investigation. Reliability 
intuitively makes sense and is easy to explain, but care will need to be taken when 
determining its impact; it will likely have significant higher-order interactions. 

• Intra-Squad Comms Delay 

The MANA manual states that comms delay “specifies the number of time steps 
that must pass before an agent’s contact information is placed onto its parent squad SA 
map” (McIntosh et al. 2007, 66). Intra-Squad Comms Delay was a fairly significant 
factor but will likely be a poor factor for use to represent impacts to communications and 
SA. Like reliability, it intuitively makes sense and is easy to explain, but when 
considering the fact that most combat squads have line of sight communications with 
each other, it is hard to justify that these types of communications could be degraded by 
adversary actions. Thus, while delay will likely be something other than zero, it will 
likely be constant, and therefore will not be included in my detailed investigation. 
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• Accuracy 

The MANA manual describes Accuracy as the “probability that a contact’s type 
will be passed correctly....an accuracy of 0% results in always sending an incorrect 
contact type while 100% means the information will always be sent as correct contact 
type” (McIntosh et al. 2007, 70). While accuracy was a significant factor it should not be 
used to model locational or positional error. Because MANA models accuracy as an 
“error of classification,” and not a positional error, it is not a good factor for my purpose, 
it is misleading. Though it has some use as an uncertainty factor to induce fratricide, I 
found that modifying it much below 95% induces large numbers of agent state changes, 
from enemy to friendly and back again. Thus, I will not use this as an input factor. 

• Capacity 

The MANA manual describes capacity as the “number of messages that can be 
sent through the link per time step” (McIntosh et al. 2007, 70). Capacity was the least 
significant factor and is a poor factor for representing the impacts to communications 
from a D3SOE. Because MANA models communications links individually, with no 
common links, this factor serves to only limit the number of messages per time step, per 
link. If 10 contacts are made in a time step, and there are four outbound links, a capacity 
of 10 for each link would be needed to complete the update for that time step. But this is 
not how HQs disseminate SA to subordinates, which is done over a single digital link 
simultaneously. Thus, capacity does not accurately represent actual communications, and 
it will not be considered in further investigations. 

ISR Collection Degradation 

Here I took advantage of the fact that for Intel to reach the forces that need it, it 
must travel through a communications network. Thus, I can degrade the info link piece in 
conjunction with the communications degradation in the exact same manner as done 
previously. To induce impacts from the loss of ISR capabilities itself, I can do this in two 
ways. First, for systems with representative agents in the model such as UAVs, HAASs, 
and SmallSats, it is possible for the adversary to kill them, and thus the degradation of the 
system will be induced into the model directly. Second, for systems that do not have an 
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agent representing it directly, such as higher HQ SIGINT, ISR, and other higher echelon 
UAVs, I can simply create a separate sensor at the HQ element that can observe the entire 
battle space to represent each of these Intel sources. Then, by manipulating the sensors 
probability of detection or time between detections, I can represent various degrees of 
degradation of the specific collection asset. Thus, factors which I identified as being 
likely surrogates for representing these effects were probability of detection and time 
between detections. 

• Detection Range-Time Table 

The MANA manual describes detection range as “the average amount of time 
between detection events for ranges r less than or equal to the specified range R (and 
greater than the next lowest range, if specified)” (McIntosh et al. 2007, 58). Detection 
range was a fairly significant factor and seems to be a good factor for use to partially 
model the impacts from degraded ISR collection capabilities or the loss of these systems. 
This factor becomes increasingly important the more dependent operations are on higher 
UAV, ISR, and SIGINT collection capabilities. Since this intelligence directly impacts 
the success of higher HQ and local fire support, I expect a significant impact to 
operations as intelligence collection is degraded. Therefore, I believe that this will be a 
fairly significant factor, so I will keep it for further investigations. 

• Classification Range-Probability Table 

The MANA manual describes classification range as “the probability of acquiring 
a target for classification purposes once it has been detected on a given time step for 
ranges of less than or equal to the specified range (and greater than the next lowest range 
if specified)” (McIntosh et al. 2007, 58). Like detection range, classification range was a 
fairly significant factor and seems to be a good factor for use to partially model the 
impacts of degraded ISR collection capabilities. But these impacts can also be 
represented in the average time between detections, and rather than manipulating two 
factors to represent the same effect, I decided to keep this factor constant and modify the 
average time between detection. Thus, this factor will no longer be considered. 
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PNT Degradation 

Here, there was no direct correlation to any factors that could induce the type of 
degradation I was looking for. MANA does not have a method to induce positional error 
or represent location uncertainty by agents. Once an agent is detected and classified, its 
location is known without error. Additionally, all agents always know exactly where they 
are. In reality this is never the case, and in a D3SOE it is even further from the truth. 
Unfortunately, agents have perfect SA, at least for what they have detected or have been 
told about. So the question is how can we trick the model into demonstrating behaviors 
that can mimic locational inaccuracies? This must be considered in a few different ways. 
First, for movement, one must understand that most ground forces have redundant maps 
and compasses that they can use in conjunction with terrain recognition to navigate. 
Thus, while not optimal, and likely slowing the decision making process and pace of 
battle, ground forces can still maneuver. So it is easy to argue that the maneuver speed of 
these forces, which is a factor we can manipulate as a surrogate, will be slower, at least 
while not in contact. Secondly, for fires, the only fires that are affected by GPS 
inaccuracies are indirect fires like artillery and deep strike capabilities. All direct fire 
forces can physically see their adversaries, and are not affected by locational errors at all. 
Thus, for the indirect fire systems, likely surrogate factors that we can use to represent 
locational and targeting inaccuracies include the munitions Ph, the blast radius, or the 
number of hits to kill of the target. 

• Movement Speed 

The MANA manual describes movement speed as “the number of cells an agent 
can move in a given time step” (McIntosh et al. 2007, 51). Movement speed was a 
significant factor and seems to be a moderately appropriate factor for use for partially 
representing the impacts from location and positional error. Positional error would likely 
induce uncertainty, and thus slow the speed of movement as decision makers are forced 
to collect more information to account for the increased uncertainty. Care will need to be 
taken when determining the impacts and should be done by squad by mission, based on 
proximity to targets. Ground forces should be the least impacted, while artillery, who are 


296 



semi-dependent on position and target accuracy should be more impacted, and deep 
strike, precision weapons systems, aviation, and UASs should be the most impacted. 

• Range/Hit Rate per Discharge 

The MANA manual describes hit rate per discharge as “the hit rate per firing of 
the weapon for specified ranges” (McIntosh et al. 2007, 62). Hit rate per discharge was a 
fairly significant factor and seems to be a good factor for partially representing the 
impacts from degraded targeting information for indirect fires. Hit rate per discharge 
becomes increasingly important as operations become more dependent on higher level 
fire support. As more and more preparatory and support fires are utilized, the bigger the 
impact will be in a D3SOE. Therefore, I think this is a fairly significant factor and will 
likely have several higher-order interactions, so I will keep it for further investigation. 

In conclusion, based on this analysis, the following surrogate input factors (from 
this point forward referred to simply as factors) will be used in my analysis. First, for 
communications degradation, I have chosen to ignore accuracy, intra-squad delay, and 
capacity. I will use a combination of latency and reliability to represent communications 
degradation for all links. Next, for ISR collection degradation, I have chosen to ignore 
classification range. I will use average time between detections (detection range) to 
represent the impacts of degraded ISR collection on collection assets. Finally, for PNT 
degradation, I choose to keep both factors. I will use movement speed to represent 
degradation of positional and destination accuracy for all moving agents, as well as hit 
rate per discharge to represent the impact of degraded targeting data for all indirect fire 
agents. The combination of these five input factors should give me adequate capacity to 
represent the effects and impacts from a D3SOE on combat operations. 

c. Potential Output Responses for Combat Effectiveness 

With potential input factors identified, it is now time to investigate the available 
output responses MANA provides in order to determine its feasibility for use in my 
research. Because MANA is a combat model, its outputs naturally include a host of 
relevant combat statistics to include force losses, losses by source, as well as killer- 
shooter information. Thus, it is easy to generate measures of combat effectiveness for 
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comparison, such as FER, LER, and EoB to name just a few. Because this is a 
exploration, I will be looking at several potential output responses in order to investigate 
their potential for accurate representation of combat effectiveness. Thus, it seems MANA 
is well equipped to provide adequate output data needed to enable a large scale DOE and 
drive meta-model development. 

d. Model Development 

Eor this exploration, I started with a toy model provided by Prof Jeff Appleget in 
his Combat Modeling Course. I then expanded it to include specific agents from the 
works of Treml (2013) and Soh (2013), specifically the Mechanized Infantry and Attack 
Aviation from Treml and the UAVs from Soh. I then significantly expanded the model, 
agent behaviors, capabilities, and other factors to loosely represent a potential scenario 
based on my understanding of the new AOC. Through trial and error over the course of a 
month, I completed over 20000 development simulation runs until I reached a point 
where I believed the agents in the model were mostly acting as expected, and the output 
was about what I would expect. This face validation would be sufficient for this initial 
investigation, and yielded a force-on-force combat simulation where the Blue forces are 
attacking a Red force. 

Red forces are fighting from well prepared positions, are comparably equipped 
and trained, and are on their own home turf. The Blue force was comprised of a Battalion 
(-I-), comprised of Armor, Mechanized Infantry, Anti-Tank, Infantry, Artillery, Aviation, 
and fires/intel support from higher, for a total of 140 agents. The Red forces were 
comprised of a similar mix of forces minus the aviation. Mechanized Infantry, and UAV 
assets, with the addition of Air Defense Artillery (ADA) assets, for a total of 52 agents. 
Therefore, the Blue force enjoys roughly a 3-1 military advantage. Thus, roughly 
speaking, with all things equal, even though Blue forces do have a better and more 
capable force structure, we should expect the Blue force to win about 50% of the time. 
Eigure 100 describes the starting positions for the simulation. 
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Figure 100. MANA Screenshot 


For the victory/stop conditions, I choose to establish the following break points 
where either side would withdraw from the fight, and thus end the simulation. For the 
defending Red forces, I determined that they would fight to a break point of 40 casualties, 
or after the loss of roughly 75% of their starting strength. For the Blue forces, I set the 
break point to 70 casualties, or after the loss of roughly 50% of their starting strength. 
Additionally, I set a maximum run length of 2500 time steps (3.5 hrs.), this would call a 
tie if victory was not achieved by either side. I did this to limit situations where the 
neither side could reach victory. 

e. DOE 

Now that the exploration and development of the model was complete, it was time 
to perform a DOE to produce output in which I could conduct analysis and develop 
models to represent the impacts of varying factors on their impact to the combat 
effectiveness of the blue force. Table 71 shows the five selected design factors which will 
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be used as surrogates for representing the effects of a D3SOE on combat operations. 
These include Latency, Reliability, Movement Speed, Hit Rate per Discharge, and Time 
between Detections, and included the high and low values for each factor in the design. 


Table 71. DOE Design Eactors and Levels 


Factors 

Name 

Squad 

Location 

Tab/Link 

Setting 

LOW Value 

High Value 

Dectmiles 

Base Case 

1 

6 Inf Latency 

1 

Inter SqdSA 

All Links 

Latency 

2 

24 

0 

2 

2 

BTank Latency 

3 

Inter Sqd SA 

All Links 

Latency 

2 

24 

0 

2 

3 

B Arty Latency 

4 

Inter SqdSA 

All Links 

Latency 

2 

24 

0 

2,4 

4 

BAT Latency 

5 

Inter SqdSA 

All Links 

Latency 

2 

24 

0 

2 

5 

B HQ Latency 

9 

Inter SqdSA 

All Links 

Latency 

2 

48 

0 

12 

6 

B UAV Latency 

11 

Inter SqdSA 

All Links 

Latency 

2 

48 

0 

30 

7 

BHelo Latency 

13 

Inter SqdSA 

All Links 

Latency 

2 

24 

0 

2 

8 

B Brad Latency 

14.16,18,20,22,24 

Inter SqdSA 

All Links 

Latency 

2 

24 

0 

2,4 

9 

B Brad Inf Latency 

15,17.19,21,23,25 

Inter SqdSA 

All Links 

Latency 

2 

24 

0 

2 

10 

6 inf Reliability 

1 

Inter Sqd SA 

All Links 

Reliability 

50 

100 

2 

95 

11 

B Tank Reliability 

3 

Inter SqdSA 

All Links 

Reliability 

50 

100 

2 

95 

12 

BArty Reliability 

4 

Inter SqdSA 

All Links 

Reliability 

50 

100 

2 

95 

13 

BAT Reliability 

5 

Inter Sqd SA 

All Links 

Reliability 

50 

100 

2 

95 

14 

BHQReliabllity 

9 

Inter Sqd SA 

All Links 

Reliability 

50 

100 

2 

95 

15 

BUAV Reliability 

11 

Inter Sqd SA 

All Links 

Reliability 

50 

100 

2 

95 

16 

BHelo Reliability 

13 

Inter Sqd SA 

All Links 

Reliability 

50 

100 

2 

95 

17 

BBrad Reliability 

14,16,18,20,22,24 

Inter Sqd SA 

All Links 

Reliability 

50 

100 

2 

95 

18 

B Brad Inf Reliability 

15,17,19,21.23,25 

Inter Sqd SA 

All Links 

Reliability 

50 

100 

2 

95 

19 

B Tank Default Speed 

3 

Tangilbles 

Default State 

Movement Speed 


15 


15 

20 

BArty Default Speed 

4 

Tangilbles 

Default State 

Movement Speed 

5 

10 

2 

10 

21 

B AT Default Speed 

5 

Tangilbles 

Default State 

Movement Speed 

6 

13 

2 

13 

22 

BHelo Default Speed 

13 

Tansilbles 

Default State 

Movement Speed 

45 

90 

2 

90 

23 

B Bradly Default Speed 

14,16,18,20,22,24 

Tangilbles 

Default State 

Movement Speed 

7 

15 

2 

15 

24 

B HQ Hljther Deep Strike 

9 

Weapons 

1 

Hit Rate per Discha^e 

0 

0.8 

2 

0.8 

25 

BArty Probof Hit 

4 

Weapons 

1 

Hit Rate per Dischaiite 

0.2 

0.8 

2 

0.8 

26 

B UAV Sensor ISR Vehicle 

11 

Sensors 

1 

Detect: Ave Time Between Detections 

2 

600 

0 

2 

27 

BArty Radar 

4 

Sensors 

3 

Detect: Ave Time Between Detections 

10 

240 

0 

10 

28 

B HQ Sensor ISR 

9 

Sensors 

2 

Detect: Ave Time Between Detections 

150 

1800 

0 

150 

29 

B HQ Sensor Sigint 

9 

Sensors 

1 

Detect: Ave Time Between Detections 

120 

1800 

0 

120 


As shown in Table 71, when multiplying the five input factors across the number 
of agent squads these factors impacted, left me with a 29 factors design that would 
attempt to represent the impacts from combat operation in a D3SOE. Because I had no a 
priori regarding the model, I was interested in maximizing to the greatest extent possible 
the insight I could gain from this investigation. With no expectations for the output, 
which factors were significant, nor the level/orders of interactions (E‘ order, order, 
higher-order interactions, I decided that a NOLH design would give me the most ability 
to explore the tradespace and still be able to identify non-linearity and higher order 
interactions. With that said, the 257 DP design shown in Table 72 was created using the 
NOLHdesigns_v5.xls tool, which is available on the HARVEST website at 
https://harvest.nps.edu . 
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Table 72. NOLH Design Matrix. 

Source Naval Postgraduate School (2017). 



Running each design point 1000 times for statistical significance led to the need 
for a simulation run of 25700 iterations of the model, to include an additional 1000 runs 
of the base case scenario. The replications were done using the NPS OR departments 
computing cluster, and was completed in less than a day. The availability and rapid 
turnaround of the MANA compatible computer cluster is a significant advantage of 
MANA as an M&S package. Since MANA is used extensively in the OR department, 
there resides a host of support for the M&S package, to include pre-existing scripting to 
enable rapid model execution of large scale designs. Because of the ability to rapidly 
develop and execute a large scale DOE, it is easy to produce enough output data for 
detailed analysis. 

f. Analysis 

The purpose of this analysis is to investigate the feasibility of MANA as a 
potential M&S for my dissertation. Thus, I will not be focused on analysis per say, but 
more on “how well” MANA supports my research needs for analysis. I am trying to 
determine the feasibility and potential of MANA to support the analysis needed to 
achieve my research outcomes as outlined in Chapter I. In order to test this, the following 
analysis uses the same general procedures as I expect to follow for my final research 
model, to include a broad case investigation as well as a more detailed NOLH analysis. 
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Base Case (1000 runs) 

For my analysis of the base case scenario, I focused on developing an 
understanding of how the model functioned in a non-degraded scenario, i.e., a scenario 
where there were no impacts from adversary counter-space activities. To do this, I looked 
at two specific groups of data: the input factors (surrogates) for representing degradation 
to combat operations due to a D3SOE, as well as the potential output responses as well. 
Then, tying these two together, focused on quantifying the impacts of the input factors 
had on the response factors with regard to metrics of combat effectiveness. So let us start 
by looking at the factor responses. 

The purpose of investigating the factor responses themselves is to look for any 
additional insight into the operation of the model, as well as any other useful insights that 
could help determine MANA’s viability as a modeling package for my research. My 
interest here deals more in understanding how the model operates, how input settings 
effect output responses, what outputs are available for analysis, gaining insight that may 
help aid in my decision of an M&S package, as well as identifying any lessons learned 
for incorporation into the final model. To address these interests, I looked at the statistics 
and distributions of losses for each agent within the simulation, as well as the killer- 
victim score board. As an example. Figure 101 shows the Blue Tank casualty analysis. 
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Figure 101. MANA Tank Casualties Summary Statistics 
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As shown in Figure 101, using IMP I was able to produce detailed statistical 
analysis of output responses, in this case. Blue Tank Casualties. While this was just one 
of many available output responses, the results were consistent throughout all the 
potentials I investigated. For the most part, there was nothing significant learned that was 
not already understood from my previous analysis. There seems to be ample fidelity in 
the output data of MANA to perform a wide range of analysis, as well as to support 
model development. 

Next I looked at four potential metrics for combat effectiveness, Pv, LoB, FER, 
and LER, and then attempted to see if and how MANA could be used to generate data for 
analysis. While these may or may not be the actual metrics for combat effectiveness used 
for my research, they were more than adequate to serve in this role for the purpose of this 
investigation. Euckily, MANA is built as a combat model, so all of these potential 
metrics are easily calculated. Blue victory can be determined simply by counting the 
number of times the Red Eorce lost, i.e., the number of times red losses were >=40. EoB 
is calculated directly by multiplying the number of time steps at the simulations end by 
the size of time step. EER can be calculated directly from the outputted Red and Blue 
losses as well as their respective starting force levels. Eikewise, EER can be calculated 
directly from the Red and Blue losses. So, it seems MANA is fully capable of providing 
adequate data for analysis of combat effectiveness. Eigure 102 shows the base case 
statistics from IMP for Blue victory. 
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Mean 0.946 

StdDev 0.2261308 

Std Err Mean 0.0071509 
Upper 95% Mean 0.9600325 
Lower 95% Mean 0.9319675 
N 1000 


Eigure 102. MANA Summary Statistics of Exploration Run 
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As shown in Figure 102, during the base case run of my model, Blue forces won 
roughly 95% of the time, with a 95% Cl of (0.932, 0.960). Regardless of the fact that 
Blue forces won more than I was expecting, the model outputted adequate data to suggest 
that Victory could be used as a potential metric for combat effectiveness. Unfortunately, 
by itself, victory does not provide a lot of insight that most decision makers would find 
useful. For example, due to its discrete nature, victory does not lend itself well to 
understanding the nuances of variance needed for informed COA comparison, which is 
the primary purpose of this dissertation. While victory is important, how do you compare 
one victory to another? Therefore, at best it can serve as a secondary measure of combat 
effectiveness used in conjunction with another more quantifiable metric, but will likely 
not be useful by itself other than for screening. 

LoB is another metric that does not lend itself directly to decision support. While 
LoB is arguably an important metric for operational decision support, it should not be 
considered a primary metric for evaluating combat effectiveness. As with victory, LoB is 
not useful by itself, and only becomes important when combined with a more insightful 
metric for evaluating combat effectiveness. Simply put, one does not decide on a COA 
based on the LoB alone. One must base it on first achieving some other primary metric of 
combat effectiveness, and then look at minimizing LoB as a secondary comparison 
metric. Therefore, like victory, this will only be good as a secondary metric for 
quantifying the impact of D3SOE on combat effectiveness. 

Both FER and EER seem to have the potential to serve as primary metrics of 
combat effectiveness. Both deal with the interaction of losses by each side, and thus, 
gives an easily understood and quantifiable metric that can be used to compare COAs and 
alternatives. Thus, both have potential, but because EER takes in account the starting 
strengths of each side, I believe that EER offers more insight and detail as a primary 
metric of combat effectiveness than EER. Eigure 103 shows the summary statistics of the 
EER that were provided by JMP. 
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Figure 103. FER as a MANA MOE 


As with Pv, using MANA and then IMP to conduct my analysis, I was able to 
produce detailed statistical analysis of PER in order to judge its fitness for use as a 
potential metric for combat effectiveness. PER seems to provide a good representation of 
the outcome of the battle, and in quantifiable terms that are easily understood and 
conveyed to decision makers. As shown in Pigure 103, the distribution shows an 
approximately normal distribution of outcomes, slightly left skewed with a small right 
tail, which is expected when considering the high likelihood of Blue Py. The mean PER 
of 2.26 shows that for every 1% loss of Blue force structure, the Red force will lose 
2.26% of their force strength. 

In conclusion, from this initial investigation of MANA, if selected as my M&S 
package for my research, I will likely use PER as the primary metric for quantifying 
impacts to combat effectiveness, with both Py and EoB being used as secondary metrics. 
In decision support, decision makers can specify a minimum acceptable FER, and use the 
output of the model to highlight COAs that meet this requirement. 

NOLH Design (25700 Runs) 

Pollowing a detailed understanding of the base case scenario, it was now time to 
verify that MANA was capable of supporting the generation and analysis of the output 
data of a large scale DOE. The DOE would vary the five input factors across the range of 
potential settings in order to identify their impacts to output responses as well as to the 
selected metrics of combat effectiveness. Due to MANAs extensive use at NPS, the 
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execution of the entire 25700 point design took less than a day, and this DOE output data 
was then imported into IMP. Once in IMP, I tested the usability of IMP to perform: 
detailed analysis, determine factor significance, identify nonlinearities and higher order 
interactions, as well as meta-model development. I started my analysis by conducted a 
stepwise regression for screening then a least squares fit (effects screening). I set PER as 
my Y variable, and then set the five decision factors (factorial to the 2"“^ degree and 
polynomial to the 2"‘^ degree) as my model effects. This captured all main effects and 
two-way interactions, as well as any non-linearity and higher order interactions across the 
noise factors, the prediction plot and summary of fit for which can be seen in Pigure 104. 
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Pigure 104. MANA PER Prediction Plot (100 Replications) 


2 

As shown in Pigure 104, the prediction plot of PER has an adjusted R of .307, 

which most analysts would consider a weak fit. This is evident by observing the large 

amount of variability in the prediction plot. While I was expecting a better fit, it is 

important to understand that the model I built is only an exploratory model, and thus, I 

am not too concerned with the level of fit thus far. There is entirely too much 

simplification and uncertainty at this point to take anything but the most simplified and 
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high level insights from this model. Regardless, I did verify that MANA provides a rich 
enough output stream allowing for a detailed DOE to conduct higher level analysis and 
modeling of responses in JMP for use in this dissertation. This prediction of PER allowed 
me to generate the meta-model shown in Pigure 105, which ties the five input factors to a 
quantifiable measure of combat effectiveness. 


Prediction Expression 

0 47333428114372 

♦0 00043192015284 'Blue AT Latency 
--0 0014582283022 'Blue HQ Latency 
■'-0 0008105891975 'Blue Inf Reliabilrty 
"O 00019656132609 'Blue Tank Reliabilrty 
-4) 0000160197314 'Blue AT Reliability 
-0 00374272735472 'Blue HQ Reliability 

♦ -O 000008040607 'Blue UAV Reliability 
■►4) 000525623044 'Blue Helo Reliability 

■'O 00018024803954 'Blue Brad Inf Reliability 

♦ O 07122226834542 'Blue Tank Default Speed 
+ 4) 0264123611131 'Blue Arty Default Speed 
♦O 005389626275 'Blue AT Default Speed 

♦ O 00047026954033 'Blue Helo Default Speed 
■'O 00826421938212 'Blue Bradly Defaull Speed 

♦ 0 09179780459381 'Blue HQ Higher Ph/dis 

♦ O 23815054904397 'Blue Arty Ph/dis 
4)0003212558765 

♦ 

'Blue Arty Radar Ave Time bet Det 
(Blue AT Latency-13 0038910505837 ) 

■►.((Blue Helo Default Speed-67 5000778210117 ) 
['-0 0000967934007 
(Blue HQ Latency-25 0038910505837 ) 

-.[(Blue HQ Reliability-75 0001556420234 ) | 

['-0 0000389799804 J 

(Blue HQ Latency-25 0038910505837 ) 

-.[(Blue HQ Higher Ph/dis-0 4003112840467 ) 1 
['0 00026548742769 J 

(Blue Inf Reliabilrty-75 0001556420235 ) 

-.[(Blue Helo Reliability-75 0001556420235 ) | 
['-0 0000101274732 J 

(Blue Tank Reliabilrty-75 0001556420234 ) 
-.[(Blue AT Reliability-75 0001556420234 ) ] 

['-0 0000444226714 J 


Pigure 105. Meta-Model of PER (partial) 


During my research I discovered that Treml (2013) also encountered a similar 
problem where he was getting a fairly low R^. He adjusted this R^ by calculating the 
mean response of each MOP across the 460 replications, which yielded a new R of .813. 
While this method was successful in vastly improving the explanatory power of his 
model, it also removed all of the variability across his design points, which is often 
sought in combat models. Thus, while this method was good for determining the 
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expected mean FER for any give setting of the five input factors, it was no longer useful 
in capturing the uncertainty or variability. To see if this method could be used with my 
data in order to improve the model fit, I conducted a similar procedure, and took the 
means of my responses across the 100 replications at each design point and re-ran my 
analysis, which can be seen in Figure 106. 



Figure 106. MANA FER Prediction Plot (Means of Replications) 


As shown in Figure 106, by using the means of each of the 100 replication sets, I 
was able to take a poor fitting model with an adjusted R of 0.307 and transform it into an 
excellent fitting model with an adjusted R of 0.96. Again, while the meaning of the 
replications significantly hampers our ability to capture the variability of the combat 
model, it does show that a good fitting model is possible if means are more important 
than capturing the true variance. Although variability of the potential outcomes is often 
just as important as the mean, there are often times when means are sufficient. While it is 
too early to determine if this issue will impact my research, it will be important to capture 
and address it again once the final model is built. Regardless, I believe that even with the 
reduced capability of this model to capture the variability of potential outcomes it still has 

308 











some usefulness in addressing the mean likelihood, and thus, is appropriate for this initial 
investigation. Using the predietion from Figure 105, I investigated whieh input faetors 
and interaetions were the most signifieant. The sorted parameter estimates for this 
investigation ean be seen in Figure 107. 


J Sorted Parameter Estimates 


Term 


Estimate Std Error t Ratio 


Blue Tank Default Speed 

Blue HQ Reliability 

Blue Arty Ph/dis 

Blue Arty Default Speed 

Blue Arty Radar Ave Time bet Det 

Blue HQ Higher Ph/dis 

(BlueTankDefault Speed-11.0006)*(Blue Tank Default Speed-11.0006) 

Blue HQ Latency 
Blue Bradly Default Speed 

(BlueTankDefault Speed-11.0006)*(Blue Bradly Default Speed-11.0006) 

(Blue HQ Reliability-75.0002)*(Blue HQ Reliability-75.0002) 

(Blue Arty Ph/dis-0.50008)*{Blue Arty Radar Ave Time bet Det-125.004) 

Blue Inf Reliability 
Blue AT Default Speed 

(Blue Tank Reliability-75.0002)*(Blue Tank Reliability-75.0002) 

(Blue HQ Reliability-75.0002)*(Blue Tank Default Speed-11.0006) 
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(Blue Tank Reliability-75.0002)*(Blue AT Reliability-75.0002) 
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Figure 107. MANA parameter Estimates for Model Fit 


Even though this is a fairly large list of significant factors, 27 in all, it is easy to 
see that just eight factors and their higher order interactions dominate the response. Thus, 
it is possible to use this sorted list as a screening tool, then selecting a more detailed 
design, focus on just these eight factors to explore the tradespace in more depth. 
Although the factors that I expected to be significant were not, the manner in which I 
developed and implemented the model seriously undervalues the value of intelligence 
collection assets due to the limited amount of time given to build the combat picture and 
execute preparatory fires prior to combat. These are issues that can be addressed in future 
versions of the model, but regardless of the outcome of this basic analysis, it is evident 
that MANA is capable of producing robust data with enough depth to allow for valuable 
analysis in support of my research. 


309 










g. Findings 

Overall, MANA provides a robust enough environment to more than adequately 
meet the needs I have for my dissertation. Following the investigation, I conducted a 
qualitative assessment of MANA’s ability to meet each of my primary and secondary 
research needs, and the scoring of this assessment can be seen in Table 73. Following the 
investigation of all potential M&S packages, this assessment of MANA will be used to 
compare the three potential alternatives. 


Table 73. MANA Assessment Table 


Weighted Decision Matrix 

Weight 

Factors 

MANA 

6 

Ability to Simulate a D3SOE 

3 

5 

Support Availability 

5 

5 

Cluster Access 

5 

5 

Ease of Use 

5 

4 

Rapid Development 

4 

4 

Existing Models 

5 

4 

Ease of Analysis 

4 

4 

Output Data Density 

4 

3 

Learning Curve 

4 

3 

Behavior Monitoring 

4 

3 

DOE Tools Available 

5 

2 

Ease of DOE Execution 

5 

Total Value 

200 


As shown in Table 73, while I do not believe that MANA is well suited for 
modeling the impacts from a D3SOE, it was well suited to support my work across all 
other areas when considering my primary and secondary research considerations. Couple 
this with the fact that I believe that there are adequate alternative factors in MANA to 
represent all of the effects I am trying to model through the use of surrogates, and I am 
more than confident that MANA is capable of meeting nearly all my research needs 
without much additional effort. 

2. Exploration of JDAFS 

JDAFS has been in development on and off for almost a decade, and its creator. 
Dr. Amie Buss, is a member of the MOVES faculty here at NFS. The proximity of Dr. 
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Buss provides a source of resident experience and knowledge that could significantly aid 
my research. JDAFS was initially designed as a fires allocation tool but was later 
expanded to perform a much broader set of M&S tasks. JDAFS is a low resolution agent 
based simulation framework that focuses on supporting users who need a fast, flexible, 
and customizable M&S package. JDAFS nature as a DE simulation “leads to fast 
execution times, which enable the analyst to quickly explore the parameter space for the 
desired situation” (Buss and D. K. Ahner 2006, 4). I initially had little knowledge of or 
experience with JDAFS, but because of its availability and resident expertise, as well as 
the fact that it was a DE simulation, a rarity, I was more than curious about its potential in 
supporting my work. 

a. Initial Observations 

Eike MANA, my initial investigation of JDAES was focused on determining the 
usability of the model, how it operated, and how easily it could be developed and 
modified. While my initial feelings regarding JDAES as a potential M&S package were 
questionable, I believed that JDAES had enough potential to model a D3SOE that it was 
worth investigating in more depth. When it comes to usability, JDAES is not the most 
user friendly or elegant M&S package I have seen, nor does it have what I would call a 
quick learning curve, but it is fairly easy to implement and run with a simple UI that is 
easy to navigate. While it comes with a fairly detailed user manual, its nature as a 
simulation framework leaves much of the simulations development and utilization 
requirements to the user. Thus, while JDAES is extremely flexible in that a user can add 
any functionality desired, this flexibility comes with a cost, forcing a user to code nearly 
all of this functionality into the model. While this could be mitigated by the re-use of 
existing models that have the functionality I was interested in, there were simply none 
available at the level of complexity I needed. Thus, the use of JDAES as my M&S 
package would come with a significant amount of time and effort to prepare it for use. 

b. Potential Factors for Representing a D3SOE 

My initial investigation included reviewing the factors in JDAES that I thought I 
could use to model the impacts from a D3SOE. Because JDAES is a framework, I began 
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my review by experimenting with the most complex example model I could find to avoid 
developing an entire model just for an investigation. Like most M&S packages, there are 
no direct or specific factor settings for degrading communications, ISR, and GPS in 
JDAFS. Luckily, this makes no difference with regard to JDAFS because any 
functionality needed can be added, at least to the extent that the foundational rules of 
JDAFS allow. Thus, as long as the underlying rules are not violated, it should be possible 
to add any needed functionality and appropriate factors. While this could potentially be 
cumbersome, if the user has the resources, JDAFS seems to have ample flexibility to 
represent the expected impacts and effects of a D3SOE. For consistency, potential 
JDAFS factors will be binned as before, into three distinct groups: communications 
degradation, ISR degradation, and PNT degradation. Recall that JDAFS has no set 
factors; thus, I will discuss in more broad terms how JDAFS can be used to demonstrate 
the effects of a D3SOE on combat operations. 

Communications Degradation 

My initial investigation of JDAES regarding communications degradation led me 
to believe that JDAES may not be capable of accurately modeling the effects I needed. 
After consulting with JDAES experts, it seems that this is indeed the case, and that 
representing accurate communications and then degrading them will not be possible with 
JDAES. Because every agent in an opposing force shares the same Common Operating 
Picture (COP) without delay or inaccuracies, without actual communications links to 
degrade it will be difficult to institute any type of degradation that closely resembles 
actual combat. I would need to identify and code a means to induce error into this COP, 
to include slowing and degrading the flow of information, as well as reducing the amount 
of information shared. While this should be possible, it induces a high amount of 
uncertainty regarding the usability of JDAES to meet my needs. If the inability of JDAES 
to accurately model communications holds true, it may be unusable for my purposes. 

ISR Collection Degradation 

Because JDAES has no communications links, I was not able to degrade 
communications to represent a degradation of collection assets as done previously. 
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Detections made by an agent in JDAFS are known instantly by all friendly forces, 
without error, and there does not seem to be a way to induce lag, inaccuracies, or 
degradation of the collection capability directly. Thus, to induce impacts from the 
degradation of ISR capabilities in JDAFS, I needed to identify alternative methods that 
could be modified to simulate the needed effects. I identified two potential ways to 
achieve the effects of ISR degradation, but these would again require abstract, counter¬ 
intuitive implementation to simulate the desired impacts. While the complexity of this 
implementation is not a show stopper, it should be understood before JDAFS is used. 

• Destroying Collection Assets 

First, for systems with representative agents in the model such as UAVs, HAASs, 
and SmallSats, it is possible for the adversary to destroy them, and thus the degradation 
of the system would be induced into the model directly. This addition to the model could 
potentially be represented as a Red ADA unit with a specialized munition to target UAVs 
or SmallSats, like an SA-18. So, it looks as if I should be capable of modeling the loss of 
the UAVs through destruction, but is it is possible to degrade the UAVs in other ways? 
For example, how can I represent a partial loss of capability? To model this functionality 
will require using other methods, likely through the degradation of sensors. 

• Degrading Sensors 

By manipulating a sensors’ probability of detection, time between detections, or 
each of these two factors’ interactions with range, it is possible to represent various 
degrees of degradation of the specific collection asset. As long as adequate and 
representative range-based sensors are used, this needed functionality should be pretty 
straightforward to model. All agents will need to be provided more realistic sensors to 
account for different probabilities-based on the type of system (vehicle/personnel), but 
again, the addition of this functionality to the model should be fairly simple. 
Additionally, for me to use this model, I would need to add in extra sensors to represent 
Battalion Intel and possibly even higher headquarters ISR and SIGINT assets and use a 
range-based probability of detection sensor. These detection probabilities could then be 
modified by some unknown amount to account for the level of collection degradation 


needed to simulate degradation of ISR systems. 
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PNT Degradation 

Like other M&S packages, there was no direct way to induce the type of 
degradation I was looking for because the SA picture of all agents in JDAFS is known 
instantly by all friendly forces once it is detected, without error. Agents have perfect SA 
of all friendly forces as well as everything that is detected. Thus, there is no way to 
directly induce positional error, so like we will look to see if we can mimic locational 
inaccuracies in other ways. The representation of PNT degradation can be done similarly 
to how we did it with MANA, namely by manipulating movement speed and targeting 
accuracy factors of indirect fire systems. 

• Movement 

Because of its nature as a DE simulation, JDAFS is somewhat limited in how it 
executes movement. JDAFS uses a value score in the move manager to determine where 
the units will attack, based on what has been detected, the Probability of Kill (Pk), and the 
subjective value of the targets within range. Unfortunately, JDAFS has no other 
personality type attributes to help determine movement other than these simple 
movement rules. While I can adjust movement speed, and use it to serve as a surrogate 
for locational degradation, JDAFS is lacking in every other way in representing actual 
combat maneuver. For example, there is no associated value to maintaining unit 
cohesion, or mass, when moving. While I could likely add this functionality given 
adequate resources, especially time, consideration must be made on the feasibility of this 
investment with regard to the work required. Additionally, the number of agent 
interactions needed to adequately model movement that can account for mass and other 
principles, to include interactions with terrain, would significantly increase the number of 
events of the model. Thus, many of the advantages of a DE model would be negated due 
to the increased complexity of the model. While these are mostly usability aspects of 
JDAFS, they must be considered when selecting an M&S package for use in my research. 

• Targeting 

Targeting in JDAFS is much more intuitive than movement, and other than the 
requirement to physically code each and every weapon-to-target range-based Pk matrix, it 
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seems to be pretty straightforward. Each engagement is based on the value assigned to 
the target type, and if there are multiple targets available in a given instance, JDAFS will 
select the highest value target. Then, after calculating range, it will look up the Pk for that 
weapon-target combo and assess whether it is a hit or not. Thus, the probability of hit rate 
per discharge seems to be a good factor for partially representing the impacts from 
degraded targeting information for indirect fires. 

In conclusion, based on this assessment, I believe JDAFS has enough flexibility to 
provide some of the factors needed for use in my research. While I will not be able to 
model communications degradation effectively, at least not directly, it should be possible 
to find some other factors that could indirectly act as a surrogate for these effects. For 
degradation of PNT, I should be able to use movement speed to represent degradation of 
positional and destination accuracy for all moving agents, as well as hit rate per discharge 
to represent the impact of degraded targeting data for all indirect fire agents. Finally, I 
will use average time between detections (detection range) to represent the impacts of 
degraded ISR collection on collection assets; although I will need to drastically increase 
these factors to account for any communications degradation that cannot be modeled 
directly. The combination of these input factors should give me a decent capacity to 
represent the effects and impacts from a D3SOE on combat operations. 

c. Potential Output Responses for Combat Effectiveness 

With potential input factors identified, it is now time to investigate the available 
output responses JDAFS provides in order to determine its feasibility for use in my 
research. Because JDAFS was designed to support analysis of the allocation of military 
indirect fires, its outputs include a host of relevant combat statistics to include force 
losses, losses by source, as well as killer-shooter information. While JDAFS’ output is 
not as user friendly as other M&S packages, everything that is needed to conduct further 
analysis is present, even though it will require a fair amount of post processing to prepare 
it for import into JMP. Thus, it seems that JDAFS should be capable of providing more 
than adequate output data needed to enable a large-scale DOE and drive meta-model 
development, as needed for my research. 
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d. Model Development 

As before, the purpose here is to look at how easy JDAFS is to manipulate, to 
create new agents, and to modify their attributes to achieve the behaviors and effects I 
seek to imitate. For this exploration, I started with a toy model provided in the example 
files that accompany JDAFS. Initially, I choose what I considered the most complex 
ground combat model available, though it was not nearly as detailed as other models I 
investigated. I set out to look at a few specific concepts within JDAFS to assess its 
capabilities as well as its strengths and weaknesses. 

Once JDAFS is executed, a UI tool opens, which allows users of JDAFS to load 
and run a model, generate output data, and do some simple execution level tasks before 
running the model. What is most notable thus far is that JDAFS does not allow one to 
make any changes to the model through the UI; it exists solely to allow a user to run a 
model. In JDAFS, all agent activities and behaviors are outlined in two other documents, 
which I am able to modify to affect agent attribute changes. The first is the 
Samplelnput.mdb database file, and the second is the xml file itself. The Access database 
file is where all changes to the model can be made easily, and then JDAFS loads the 
database file by creating an xml file. So the Access database acts as the UI for model 
development, and to implement any changes during model development either the 
database or the xml file must be updated. The Access database can be seen in Figure 108. 
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Figure 108. JDAFS Access Database 


By modifying the Access database directly, and playing with certain aspects of 
each agent, I was able to change starting locations, the number of forces, quantity and 
capabilities of munitions, Pk, as well as numerous others factors. Model development was 
pretty straightforward once a user learns where everything is in the xml file, and I started 
off by adding in a search area for the UAV to move in a random search pattern. I then 
modified the Red forces AT-10 weapon and allowed it to target the Blue UAV. Through 
trial and error over the course of a few weeks, I was able to apply numerous other 
modifications to agents in order to determine their fitness for use to demonstrate the 
impacts of a D3SOE on combat operations. I ended this development with a fairly simple 
model that did not differ drastically from the original model and was comprised of six 
Blue Improved Combat Vehicles (ICV) and a single Blue UAV, pitted against eight Red 
fires systems (tank, Arty, Mortar, AT, IFV, Dismounts, MRL), and two Red Sensors. 
Figure 109 shows the JDAFS UI mid run. 
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Figure 109. JDAFS UI Window 


As shown in Figure 109, JDAFS comes with a fairly simple UI, allowing the 
developer to run a model and generate output data for analysis. While not as graphically 
appealing as other M&S packages, it is sufficient for its purpose. The model runs quickly, 
often too fast to observe most agent behaviors, fortunately, by selecting the Verbose 
After Run box in the Simulation Parameters window, the model will run much slower, 
allowing the developer to watch agent behaviors during execution. This is extremely 
useful during validation and troubleshooting. Running the model and setting up multiple 
replications was also fairly straightforward. In this case, I ran the model 100 times, and 
the execution time was less than 1 min, with an additional 1 min to compile the output 
data and create the output database. Although the JDAFS model was less complex than 
other models investigated, the advantage of a model that runs 60 times faster is notable. 

e. DOE 

My biggest concern regarding JDAFS was whether or not it could produce the 
quantity and quality of data I needed to execute a large-scale DOE. While JDAFS can 
produce multiple runs of the same design point with different random number seeds, 
there is not a fully developed tool I can use to execute a full DOE. While there are scripts 
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available that could read a design from one file, and create the associated xml document 
for that run, they are outdated, and would need a fairly significant update to become 
viable. Even so, this method is neither efficient nor elegant. For a 257 design point 
design, this method would literally create an xml file for each of these designs and then 
run each 100 times with different seeds, producing a separate output file. This output 
would then require additional scripts to format and post-process the data before it could 
be brought into IMP for analysis. It is for these reasons, namely the lack of the scripts 
needed to run JDAFS, that I was unable to execute a full DOE during this exploration. 

/. Analysis 

The purpose of this section is not focused on analysis per say, but more on “how 
well” JDAFS supports my research needs for analysis. Unfortunately, due to the 
significant requirements needed to develop a usable JDAFS model, I was unable to 
achieve many of the analysis demonstrations shown in the previous section. Even so, I 
was able to conduct a thorough enough exploration to confirm that JDAFS does produce 
adequate output data for use in my research. Though this output data will require a fair 
amount of post processing to prepare it for import into JMP, the amount of effort needed 
to do so is minimal. An example output file provided by JDAFS can be seen in Table 74. 


Table 74. JDAFS Output Database File 
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As shown in Table 74, JDAFS produces adequate output data, and other than 
requiring more post-processing, it will be capable of producing detailed enough data for 
my work. Because I was unable to perform the large-scale DOE as seen previously, I was 
unable to produce the data required for analysis. Thus, I did not show any of the 
statistical analysis as I did in the previous section. Fortunately, once the JDAFS scripting 
issues are resolved and I am able to conduct a full DOE, it is evident that JDAFS is 
capable of producing robust enough data with enough depth to allow for valuable 
analysis in support of my research. 

g. Findings 

Overall, JDAFS is a fairly flexible environment that can be used for a host of 
modeling activities, to include modeling for my research. Following this investigation, I 
conducted a subjective assessment of JDAFS’ ability to meet each of my primary and 
secondary research needs, and the scoring of this assessment can be seen in Table 75. 


Table 75. JDAFS Assessment Table 


Weighted Decision Matrix 

Weight 

Factors 

JDAFS 

6 

Ability to Simulate a D3SOE 

2 

5 

Support Availability 

3 

5 

Cluster Access 

3 

5 

Ease of Use 

1 

4 

Rapid Development 

2 

4 

Existing Models 

2 

4 

Ease of Analysis 

2 

4 

Output Data Density 

2 

3 

Learning Curve 

2 

3 

Behavior Monitoring 

2 

3 

DOE Tools Available 

3 

2 

Ease of DOE Execution 

2 

Total Value 

99 


As shown in Table 75, I felt that JDAFS was not well suited for modeling the 
impacts from a D3SOE, nor was it well suited to support my work when considering my 
primary and secondary research considerations. While I believe JDAFS has the potential 
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to meet most of my research needs given adequate time and resources, there is a high 
level of uncertainty that must be considered if selecting JDAFS for use. 

3. Exploration of SEAS 

SEAS is an agent-based M&S package that has been in development and actively 
used by the Air Force Space and Missile Systems Center for the past 20 years. SEAS is a 
constructive, time-step, mission level M&S package that supports military operations 
research in support of “developmental planning and Pre-Milestone ‘A’ acquisition 
decisions for military space systems. SEAS has proven to be a valuable military ops 
research tool by enabling exploratory analysis of new system concepts, system 
architectures, and Concepts of Operations (CONOPS) in the context of joint warfighting 
scenarios” (TeamSEAS 2017). Other than the information listed on the DOD M&S 
Catalog, I was unable to gather much data regarding SEAS. Unlike MANA and JDAFS, 
SEAS is not openly available, and thus, I was initially unable to make any assessments of 
SEAS’ potential usability for my research. After making contact with the Air Force 
Program Officer as well as Army users, primarily in the space professions, I was able to 
get a copy of SEAS for use in my exploration. Despite the limited access to SEAS and 
lack of resident expertise at NPS, its status as an actively used M&S package within the 
DOD to investigate space and C4I systems had me more than curious about its potential 
for modeling a D3SOE. 

a. Initial Observations 

Eike the two preceding M&S packages, my initial investigation of SEAS was 
focused on getting a feel for the usability of the model, how it operated, and how easily it 
could be developed and modified. SEAS is fairly easy to implement and run, and using 
its intuitive UI and IDE make it is easy to execute incremental development. While SEAS 
comes with a detailed user manual, its nature as a simulation framework leaves much of 
the simulations development and utilization requirements to the user. So, like JDAFS, 
SEAS offers an exceptional amount of developmental flexibility but at the cost of time 
and resources needed to code the needed functionality. While this issue could be 
mitigated by the use of existing models, all models currently being used by both Army 
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and Air Force planners are classified, and thus not available to support my work. Thus, 
when considering SEAS as a potential M&S package, considerations for increased model 
development requirements and coding to prepare the model must be accounted for. 

b. Potential Factors for Representing a D3SOE 

My initial investigation included a look at the factors in SEAS that I thought I 
could use to model the impacts from a D3SOE. Because this is a framework, I 
experimented with the most complex example model I could find to avoid developing an 
entire model just for an investigation. Unlike MANA and JDAES, there are specific 
factor settings for degrading communications, ISR, and GPS in SEAS. Thus, there would 
be no need for surrogate factors to represent the impacts of a D3SOE. Unfortunately, the 
simple model that was available for me to investigate did not include the needed 
functionality, and without a much more complex and detailed model to work from, I 
would need to code this functionality from scratch. Thus, while SEAS can represent the 
impacts and effects I am looking to manipulate, I was unable to do so during this 
investigation. Regardless, while the use of SEAS could be cumbersome, there seems to 
be ample flexibility to represent the expected impacts and effects of a D3SOE if 
additional time is available for the developmental effort. Eike the previous investigations, 
potential factors in SEAS will be binned into three distinct groups, communications 
degradation, ISR degradation, and PNT degradation. 

Communications Degradation 

Through my investigation, it is more than evident that SEAS has an exceptionally 
rich and detailed capability to model communications. Unlike many other M&S 
packages, SEAS does not require the use of surrogate factors to represent impacts to 
communications, it can do so directly. Not only does SEAS model communication links 
in great detail, but it can also model the degradation of communications through the 
integration and modeling of adversary counter-space capabilities. Unfortunately, because 
SEAS is a framework and I do not have access to an unclassified model, I would need to 
code all this functionality from scratch. While SEAS seems to be the most capable of the 
M&S packages investigated with regard to modeling communications degradation, a 
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significant amount of developmental effort would be needed to build a model. Thus, 
consideration for time and resources available should be made when assessing the 
feasibility of SEAS for use as a potential M&S package, especially if an unclassified 
model cannot be found. 

ISR Collection Degradation 

As with communications degradation, SEAS is able to model ISR collection 
assets in great detail, as well as the ability to model adversary counter-space capabilities 
and effects that could impact these systems, to include kinetic and non-kinetic weapons. 
In fact, the resolution and detail of SEAS is far greater than what is actually needed, 
something that should typically be avoided, and will need to be considered later. Thus, it 
again seems that SEAS is the most capable of the M&S packages investigated, and it is 
extremely well suited for modeling the impacts to ISR due to the degradation of 
collection assets. Yet, as we saw with communications degradation, a significant amount 
of developmental effort would be needed to build a model, and must be considered when 
assessing the usability of SEAS for my research. So, as before, unless an unclassified 
model can be found, the use of SEAS may not be feasible. 

PNT Degradation 

Unlike most M&S packages, SEAS is capable of degrading the capabilities of 
U.S. PNT signals directly through the model, inducing both locational and timing errors. 
This can be done in numerous ways; everything from impacting the GPS satellites 
directly to the use of terrestrial based GPS jammers is possible. What is evident is that 
once again, SEAS provides an extremely robust environment in which to degrade the 
capabilities of space based PNT. Thus, SEAS again seems to be the most capable of the 
M&S packages investigated, and is well suited for modeling the impacts to PNT due to 
the degradation from operations in a D3SOE. Unfortunately, as we saw previously, it 
would again require a significant amount of time and effort in order to build a model. 

In conclusion, based on this analysis, I believe SEAS has significant flexibility 
and depth to more than adequately represent the effects in which I am trying to simulate 
in my research. SEAS allows the user to model the systems, counter-space capabilities. 
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and effects on communications, ISR collection, and PNT degradation directly, without 
modification or the use of surrogates. This is extremely valuable to my work, and a 
serious consideration when selecting an appropriate M&S package. The combination of 
these factors gives me the best capacity to represent the effects and impacts from a 
D3SOE on space systems and space operation. 

Unfortunately, I am not interested on the impacts of a D3SOE on space systems, I 
am interested on the impacts they have on metrics of combat effectiveness, from the 
perspective of the ground force. Thus, the ability of SEAS to translate these space 
operational impacts into impacts on ground force is important. While everything 
regarding SEAS discussed thus far shows a significant capability of SEAS to model a 
D3SOE, it is important to understand that this is from an Air Eorce perspective, whose 
measures of combat effectiveness differ greatly from Army measures of combat 
effectiveness. Therefore, caution is advised when considering SEAS as a potential M&S 
package. If SEAS is selected, verifying that ground combat operations are modeled 
correctly and to the depth and level of detail needed will be essential. Eikewise, verifying 
that the impacts to combat effectiveness are adequate and using measures of operational 
effectiveness relative to ground combat will also me necessary. 

c. Potential Output Responses for Combat Effectiveness 

With the potential of SEAS to provide adequate input factors to represent 
communications, ISR collection, and PNT degradation established, it is now time to 
investigate the available output responses SEAS provides to determine its feasibility for 
use in my research. Because SEAS was designed to support analysis and comparison of 
emerging space systems and technologies in terms of impacts to Air Eorce measures of 
combat effectiveness, its outputs include a host of relevant combat statistics. These 
statistics include responses like force losses, losses by source, as well as a host of space 
and communications specific information not typically covered in other M&S packages. 
The SEAS output files are organized fairly well, and require only minor modifications to 
get them in a form suitable for importing into IMP. There is more than ample information 
available through the various output files to conduct analysis to the level that is required 
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for my work. In fact, there is more than enough information to conduct more detailed 
analysis, specifically regarding factors of communications and C4I. Thus, SEAS should 
be capable of providing more than adequate output data needed to enable a large scale 
DOE and drive meta-model development as needed for my research. 

d. Model Development 

SEAS uses a fairly well developed UI that allows a user to load, run, generate 
output data, and to do some simple execution level tasks before running the model. Eike 
ID AES, SEAS does not allow a user to make any changes to the model through the UI; it 
exists solely to allow for the running of the model and generation of output data. While 
this is arguably only a minor inconvenience, it does impact the usability of SEAS, 
specifically when considering the significant amount of iterative development that will be 
needed to build the model. All agent activities and behaviors in SEAS are outlined in a 
.war file, and modification or incremental development of the model is done through the 
Eclipse IDE. The SEAS Eclipse IDE can be seen in Eigure 110. 
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Eigure 110. SEAS Eclipse IDE Window 
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As with MANA and JDAFS, the purpose here is to look at how easy SEAS is to 
manipulate, to create new agents, and to modify their attributes to achieve the behaviors 
and effects I am looking to imitate. Because I was somewhat concerned about the ability 
of SEAS to model ground combat operations, I choose to look at two different types of 
models. Eirst, I looked at a ground maneuver model that was provided in the example 
files that accompany SEAS to investigate how SEAS modeled ground combat. Second, I 
looked at a more traditional SEAS model, which included UAVs and Radar sites, to 
investigate the models capacity regarding collection assets and communications. By 
modifying the .war file, and playing with certain aspects of each agent, I was able to 
change starting locations, the number of forces, quantity and capabilities of munitions, Pk, 
as well as others factors, which were all pretty straight forward in the well-organized and 
documented examples used. 

Through trial and error over the course of a few weeks, I was able to apply 
numerous modifications to agents to determine their fitness for use. I ended this 
development with a pair of fairly simple models. The first was the Air model, which did 
not differ drastically from the original model, and was comprised of a Blue UAV, a 
squadron of Blue strike fighters, pitted against three ADA sites. One can see a screenshot 
of the Air simulation UI in Eigure 111. The second was the ground model, and it was 
altered significantly to comprise of a four Blue tank companies executing a flanking 
maneuver against three defending and better armed Red tank companies. 
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Figure 111. SEAS UI 

As shown in Figure 111, the SEAS UI is similar to that of MANA, with the only 
significant difference being that the model developer cannot make changes to the model 
directly from this UI. Also, the SEAS UI allows the user to visualize agents at all 
altitudes, to include space-based agents like satellites or spacecraft. Eike JDAFS, any 
developmental model changes must be made directly in the .war file through the use of 
the Eclipse IDE. Running the model and setting up multiple replications was also fairly 
straight forward. For SEAS, I ran each of the two original models 30 times, and the 
execution time was noticeably fast, though any insight into this should be treated with 
suspicion because the model used here was far less complex than the other models I 
investigated. 


e. DOE 

I am fairly concerned regarding the ability of SEAS to support the execution of a 
large-scale DOE. While SEAS can be used to produce multiple runs of the same design 
point with different seeds, there is not a readily available tool I can use to execute a large- 
scale DOE. Even more concerning is that SEAS is not used at NFS, and there is no 
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resident expertise here that eould support the needed development of seripting language 
to implement SEAS on the computer clusters. Thus, SEAS would likely require a yet 
undetermined amount of time and money to contract experts to program and implement 
the needed scripting language, which could potentially induce a whole range of (delay, 
contracting, cost overrun, oversight) usability issues. It is for these reasons, namely the 
lack of execution scripts, that I was not able to do execute a full DOE as seen previously. 

/. Analysis 

As before, the purpose of this section is not focused on analysis per say, but more 
on how well SEAS supports my research needs. Unfortunately, due to the significant 
requirements needed to develop a usable SEAS model as well as the scripts needed to 
execute a DOE, I was unable to achieve many of the analysis demonstrations shown in 
the MANA section. With this said, I was able to conduct a thorough enough exploration 
to confirm that SEAS does produce adequate output data for use in my research. While 
SEAS has a much more robust set of output than most M&S packages, it is not as 
organized as I would like, and would need some post processing before it can be brought 
into IMP for analysis. Eigure 112 shows just one example of a SEAS output file. 
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Eigure 112. SEAS Weapon Output Eile Eormat 


As shown in Eigure 112, this weapons output .csv file contains all the data needed 
to compute a victim/shooter score card, and thus, it should be possible for SEAS to 
produce data to the same level of detailed needed for my work. Once the necessary 
scripts are developed and post processing is complete, analysis of SEAS output should be 
relatively straightforward. 
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Although these SEAS models were relatively simple, I was able to verify that 
SEAS is eapable of producing robust enough data with enough depth to allow for 
valuable analysis in support of my research. As long as the potential issues regarding the 
development and implementation of the necessary scripts needed to execute a large-scale 
DOE are addressed, I am more than confident that SEAS can produce the data needed to 
conduct the level of analysis needed for my work. 

g. Findings 

Overall, I believe that SEAS is an extremely flexible environment that can be 
used to meet all of my modeling requirements. Eollowing this investigation, I conducted 
a qualitative assessment of SEAS’ ability to meet each of my primary and secondary 
research needs, and the scoring of this assessment can be seen in Table 76. 


Table 76. SEAS Assessment Table 


Weighted Decision Matrix 

Weight 

Factors 

SEAS 

6 

Ability to Simulate a D3SOE 

5 

5 

Support Availability 

2 

5 

Cluster Access 

2 

5 

Ease of Use 

3 

4 

Rapid Development 

2 

4 

Existing Models 

2 

4 

Ease of Analysis 

3 

4 

Output Data Density 

4 

3 

Learning Curve 

4 

3 

Behavior Monitoring 

3 

3 

DOE Tools Available 

2 

2 

Ease of DOE Execution 

2 

Total Value 

140 


As shown in Table76, I felt that SEAS was more than capable of modeling the 
impacts from a D3SOE, as well performing fairly well across the primary considerations 
for my work. Unfortunately, SEAS did not score as well across the secondary 
considerations regarding usability, which could impact the overall utility of SEAS when 
considering my research. 
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F. M&S COMPARISON AND EVALUATION 

With the investigation complete, it was now time to qualitatively compare each of 
the three potential M&S packages against each other. Specifically, I will be considering 
how well each of the three M&S packages supports the three primary areas of a D3SOE 
which I am attempting to model, as well as how each addresses the primary and 
secondary considerations mention in the first part of this chapter. After doing this 
comparison, ample qualitative data should be available to justify the selection of an M&S 
package for use in my work. 

I. Comparison 

The following comparison is a qualitative assessment of each of the three 
potential M&S packages’ ability to meet the major requirements for my work, as well as 
their ability to meet specific secondary considerations, when compared to each other. I 
attempted to do this comparison in the order presented in the previous investigation 
sections, specifically focusing on the considerations that I thought were the most relevant. 
The following areas were considered for this comparison. 

a. Modeling a D3SOE 

When considering how well each M&S package models a D3SOE, I focused on 
how well each model supported the modeling of the three primary effects which I was 
trying to represent, specifically, communications, ISR, and PNT degradation. My 
comparison of each of the three potential M&S packages follows. 

(I) Communications Degradation 

Eike most M&S packages I investigated, there are no direct or specific factor 
settings for degrading communications, ISR, and GPS in MANA or JDAES. Euckily, 
because of the implementation of MANA, there are several factors which can be used to 
degrade communications. MANA offered many of these knobs, like latency, capacity, 
reliability, and delay, but JDAES does not. Because each agent in MANA shares 
information through links, it is easy to build accurate communications links to represent 
real world communications, then later degrade these links based on the expected impacts 
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from adversary jamming. This is not possible in JDAFS because it inaccurately models 
communications, a consideration that cannot go unnoticed. So, unless I can figure out a 
way to represent this effect, JDAFS may not be suited for modeling communications. 

Unlike MANA, which required the use of surrogate factors to represent impacts to 
communications, SEAS can do so directly. This is a significant advantage, and allows 
SEAS to model communication links in great detail, as well as the effects of degradation 
to communications and adversary counter-space systems and capabilities, something 
neither MANA nor JDAES can do. Unfortunately, the flexibility of SEAS to model these 
aspects comes at a significant cost of development time and resources needed to code in 
the desired functionality. While MANA is not as capable of modeling communications 
degradation to the level of detail as SEAS, it does not require any additional resources to 
achieve a usable model, something that must be considered. 

(2) ISR Degradation 

Once again, it seems that SEAS is able to model ISR collection assets in far 
greater detail than either MANA or JDAES, as well as the ability to model adversary 
counter-space capabilities and effects that could impact these systems, to include kinetic 
and non-kinetic weapons. But again, this flexibility requires a significant commitment of 
resources to code in the desired functionality, something both SEAS and JDAES must 
consider. While MANA is not as capable as SEAS in modeling ISR degradation, it 
requires little effort to achieve a usable model, and it is far more capable than JDAES, 
which does not have the capacity to induce lag, inaccuracies, or degradation of collection 
capabilities. Thus, SEAS is the most capable and should be a user’s first choice given 
adequate time and resources. In any situation where these resources are restricted, 
MANA quickly becomes more advantageous due to its reduced development 
requirements. In neither case is JDAES more advantageous with regard to ISR 
degradation 

(3) PNT Degradation 

Eike many other M&S packages, there was no direct way to induce the locational 
degradation I was attempting to model in either MANA or JDAES. Eor MANA, there 
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simply were no factors that could induce positional error or represent location uncertainty 
by agents. In MANA, once an agent is detected and classified, its location is known 
without error, and then communicated to other agents over time. In reality this is never 
the case, and in a D3SOE it is even further from the truth. So unfortunately, agents have 
perfect SA of all detected agents. This problem is even worse for JDAFS because the SA 
picture of all agents in JDAFS is known instantly without the need for communications 
by all friendly forces once it is detected, without error. Thus, agents have perfect SA of 
all friendly forces as well as everything that is detected. Thus, there is no way to directly 
induce positional error in JDAFS, nor is there a means in which to affect it through 
communications links, because there are none. Fortunately, unlike MANA and JDAFS, 
SEAS is capable of degrading the capabilities of U.S. PNT signals directly, inducing both 
locational and timing errors to either the satellite or to the terminal users, which is 
extremely advantageous. 

(4) General Comments 

In conclusion, it seems that SEAS is by far the most capable of the M&S 
packages investigated regarding capacity to model a D3SOE. As an Air Force model, 
SEAS was designed primarily for this purpose, and thus, can directly model all of the 
effects that I am trying to represent without the use of surrogates as I am required to use 
to various extents with both MANA and JDAFS. SEAS allows the user to not only model 
the systems, but the counter-space capabilities, and effects on communications, ISR 
collection, and PNT degradation as well, directly, without modification or the use of 
surrogates. Therefore, SEAS has the potential to be far more accurate when compared to 
other models of equivalent resolution and detail. 

MANA on the other hand meets nearly all of my primary and secondary 
requirements, but does not have the depth of factors seen in SEAS that I would need to 
model the impacts of a D3SOE on combat effectiveness directly. Fortunately, because 
MANA provides an extremely robust communications architecture and SA picture, it 
provides more than enough knobs to turn to represent the impacts from operations in a 
D3SOE through the use of surrogates. So the lack of fidelity in how MANA models a 

D3SOE is not a very significant disadvantage, far less significant than the 
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disadvantageous seen in JDAFS. While JDAFS has enough flexibility to provide some of 
the factors needed for use in my research, its inability to model communications 
effectively is a significant concern. When coupled with the unknown resource 
requirements needed to code in the desired functionality, the overall usability of JDAFS 
to support my work is significantly undermined. 

b. Usability 

With respect to my secondary consideration, MANA is extremely user friendly, 
with an exceptional amount of local support material and expertise here at NFS. MANAs 
nature as a time step model makes it extremely easy to do incremental development 
changes and conduct small simulation batch runs to observe the behavior of the model. 
JDAFS is not nearly as user friendly as MANA, although it does have a fair amount of 
support material and support expertise locally here at NFS, it is not nearly as robust as 
what is available for MANA. Although it lacks the elegance of MANA, it is fairly easy to 
implement and run, with a simple UI that is easy to navigate. What is most notable thus 
far is that JDAFS does not allow a user to make any changes to the model through the UI; 
it exists solely to allow a user to run a model. This is vastly different from MANA, which 
allows changes to the model directly from the UI which significantly reduces the 
complexity of incremental development. Likewise, SEAS is also not as user friendly as 
MANA, though it is far better than JDAFS. Like JDAFS, SEAS requires the use of a 
separate IDE for modification to the model. Thus, we cannot make developmental 
changes to the model through the UI. While not nearly as user friendly as the MANA UI, 
which provides a single user friendly UI for model development, test, and evaluation, it is 
far cleaner and more usable than JDAES. Although this is relatively insignificant by 
itself, when coupled with the lack of any local support or expertise here at NFS we find 
that the overall usability of SEAS is significantly degraded. 

c. Flexibility 

MANA is a proprietary software suite made available for use at NFS. Thus, the 
functionality of the software is limited by its development and source code, and cannot be 
expanded or changed to account for new concepts without support from the developer. 
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Thus, MANA is only as flexible as its design, and while its design is fairly flexible and 
can account for most user needs through a broad set of factors, it cannot account for 
everything. Thus, my ability to use MANA to model the entire range of a D3SOE is 
limited. So, while MANA is easy to use, highly capable, with little to no developmental 
requirements, it cannot get any better than it already is. SEAS and JDAES on the other 
hand have the potential to be far better at modeling a D3SOE. 

Because of the inherent flexibility of both SEAS and JDAES, stemming from 
their status as a simulation frameworks, and ability for growth of both is far superior to 
most commercially available M&S packages, to include MANA. Thus, because the 
source code is free and open, and does not require the owner to implement changes, both 
SEAS and JDAES are expandable, and can account for behaviors that MANA cannot. 
This allows the developer to add in any functionality they may require. This is extremely 
advantageous when considering the modeling of a D3SOE as in my research, because this 
functionality does not currently exist in most M&S packages. 

Unfortunately, this flexibility comes with the significant cost of time and 
resources in order to code in the needed functionality. Neither SEAS nor JDAES start 
with any functionality, so while they have the potential to provide a more in-depth and 
detailed accounting for a D3SOE than MANA, a significant investment of time and 
resources would be needed in model development in order to code the needed 
functionality. While this disadvantage could be mitigated by the re-use of existing models 
that had the functionality I was interested in, neither SEAS or JDAES had any available 
at the level of complexity I need. Because I do not have the time or resources needed to 
execute this development, I do not believe that SEAS or JDAES are feasible for my 
needs. And even though MANA has the potential to be less accurate than SEAS or 
JDAES, it is usable now, without development delays. Couple this with the fact that 
MANA is fairly capable, though not as capable as SEAS, and it is easy to justify the 
advantage of MANA over both SEAS and JDAES. 
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d. Support 

When it comes to support, MANA is by far the best M&S package available for 
use in my work. MANA has been extensively used in research at NPS, especially in the 
OR department, where it has been used as the primary M&S package in over twenty 
thesis and at least three dissertations. This has two significant advantages. First, it 
provides a large pool of past work from which I can uses as a foundation for my own 
work. Second, it provides a large pool of experts which I can contact for support when 
needed. By having resident experts in MANA here at NPS, any issues encountered during 
model development can be overcome, something neither JDAFS nor SEAS can claim. 

While JDAFS does have some support at NPS, it is just a single person, and not 
nearly as robust as what is available for MANA. This is further complicated by the fact 
that there has been little past work done using JDAFS, and thus, there are few models in 
which I can leverage for my own work. The support available for SEAS is even more 
restricted due to the complete lack of any resident expertise at NPS. Thus, the use of 
SEAS would likely require the contracting of outside support to achieve a usable model. 
Coupled with the fact that almost all models using SEAS are classified, and we again find 
a situation where there is little past work in which I can leverage for my own work. Thus, 
like JDAES, SEAS is at a significant disadvantage when compared to MANA when 
considering the availability of support. 

e. DOE 

Because the production of meta-models is a primary consideration for my work, 
the use of a large-scale DOE must be considered when conducting this assessment. While 
all three M&S packages were capable of executing of producing multiple runs of the 
same design point with different random number seeds, not all were as capable of being 
executed in conjunction with large-scale DOEs. Without the ability to rapidly develop 
and execute a large scale DOE it will be difficult to get enough output data for adequate 
analysis, and thus a consideration when selecting an appropriate M&S package. 

MANA is by far the simplest M&S package to use in conjunction with a large- 
scale DOE. Since MANA is used extensively at NPS, especially in the OR department, 
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there resides a host of support for MANA, to include pre-existing scripting to enable 
rapid model execution of large-scale DOEs. For example, using the NFS computer 
clusters, I was able to run a fairly complex MANA model in a 25700 design point NOLH 
design in less than a day. The availability and rapid turnaround of the MANA compatible 
computer cluster is a significant advantage of MANA over both SEAS and JDAFS, 
because the same level of support may not be available for other M&S packages. 

The lack of any historical use or expertise of SEAS at NFS has resulted in a 
situation where the required execution scripts currently do not exist. Thus, a significant 
amount of effort would need to be dedicated to the scripting and integration coding 
required to execute a SEAS model on the NFS cluster. And since it has never been done 
before, there is a lot of uncertainty regarding the ease in which this could be 
accomplished. JDAFS on the other hand does have some execution scripts available, but 
have not been used in many years and would need a fairly significant update to become 
viable for my use. Thus, another significant chunk of time and effort to be dedicated to 
the scripting and integration coding required executing either SEAS or JDAFS on the 
NFS cluster. This is a significant disadvantage when compared to the easy of a DOE 
using MANA, and could significantly degrade the usability of SEAS or JDAFS. 

/, Analysis 

Because MANA and SEAS are both combat models, their outputs naturally 
include a host of relevant combat statistics to include force losses, losses by source, as 
well as killer-shooter information that is critical for good analysis. Thus, it is easy to 
generate measures of combat effectiveness for comparison, such as FER, EER, and EoB 
with either MANA of SEAS. While SEAS seems to have a much more robust set of 
outputs than MANA, specifically regarding space and communications specific 
information, it is not nearly as well organized, and like JDAFS, will require some post 
processing before it can used in analysis. Regardless, it seems MANA and SEAS are both 
well equipped to provide adequate output data needed to conduct higher level analysis 
and modeling of responses. 
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JDAFS on the other hand was originally designed as an optimization tool, and 
thus, does not provide output data nearly as organized as either MANA or SEAS. Though 
everything that is needed to conduct further analysis is present, it will require a fair 
amount of post processing to prepare it for further analysis. One advantage of JDAFS 
over both MANA and SEAS is the fact that it is a DE model, and thus, it will not suffer 
from the same artifacts and anomalies that time-step models suffer from. While time-step 
models have many advantages, the one overarching disadvantage is that they suffer from 
induced error caused by the selection of the time-step size. Because JDAFS is a DE 
model, which executes events as they occur, it avoids inducing this type of error, and 
thus, has the potential to be far more accurate when compared to both MANA and SEAS. 

g. Agent Behavior 

The primary concern regarding agent behaviors resolve around two primary 
topics, which include accuracy of the combat model, how well a model represent ground 
combat operations, a primary consideration; and autonomy, how agents execute decision 
making and movement on the battlefield. Eet us start with combat operations. 

Because MANA is a ground combat model, it does a fairly good job at modeling 
combat operations. Although it does not model the space element in any way, as SEAS 
does, it is possible to represent the effects of space. SEAS on the other hand does not 
model ground combat operations to the same resolution or accuracy as it does the Air and 
Space component. Therefore, it is difficult to validate the performance and accuracy of 
ground combat in SEAS because not only is the accuracy of the ground segment suspect, 
but the Air Force centric measures of combat effectiveness used by SEAS do not translate 
well to Army metrics of combat effectiveness. JDAFS on the other hand, while more 
combat oriented than SEAS, is also restricted in its capabilities to its implementation as 
an optimization tool. And although it is more accurate, its range of usability is much 
reduced when compared to either SEAS or MANA. 

With regard to autonomy, MANA is the only truly semi-autonomous agent-based 
simulation I investigated. Because neither SEAS nor JDAFS possess any agent 
personality attributes that are used by the agents to make operational decisions, agents in 
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SEAS and JDAFS are not nearly as autonomous as agents in MANA. Thus, these agents 
require more directed and scripted orders from higher elements to execute decision 
making. The inability of agents to act semi-autonomously significantly reduces the power 
of the simulation to address emergent behaviors, as well as increasing the amount of 
hierarchical structure required to represent the BMC2 aspects of the simulation. 

The largest impact this has on a simulation is that almost all maneuver in SEAS 
and JDAES is scripted, and thus difficult to model the fluidity of how actual ground 
maneuver develops and transforms as the COP is developed. This is possible in MANA 
because agent decisions are based on a set of attributes that can be used to determine its 
actions. Thus, MANA does not require a script for its agents to follow as does SEAS and 
JDAES, which use a simple set of movement rules and a list of waypoints to execute 
movement. Thus, Blue forces in JDAFS and SEAS will go wherever the highest value 
target is, regardless of the efficiency of the move or the location of friendly forces or 
other objectives. This lacks in almost every way when compared to MANA in 
representing actual combat maneuver. For example, there is no associated value to 
maintaining unit cohesion, or mass, when moving in either JDAFS or SEAS without 
scripting. The inability of JDAFS and SEAS agents to model these standard principles of 
military tactics is a significant drawback. 

Another significant disadvantage of JDAFS regarding movement is that it does 
not account for terrain or Fine of Site (EOS) calculations during movement. This was by 
design, and due to the fact that JDAFS was originally developed to model the allocation 
of fires and sensors, both typically considered Non-EOS systems, and thus, not impacted 
by intervening terrain or EOS restrictions. But because I am attempting to use this model 
in support of ground combat operations, JDAFS inability to model the impacts from 
terrain is a significant disadvantage. 

2. Evaluation 

With the investigation and comparison complete, I decided to do a weighted 
decision support matrix to attempt to quantify how the three potential M&S packages met 
my primary and secondary requirements for my research. While this is highly qualitative. 
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because I compared each potential M&S package against each other, as well as assessing 
the ability of each to meet both primary secondary usability considerations, I am 
confident that it is more than appropriate for this purpose. Table 77 shows the weighted 
decision matrix I used to conduct my comparison, where I weighted each factor by its 
expected impact and importance to meeting my research objectives. 


Table 77. Decision Matrix 


Weighted Decision Matrix 

Weight 

Factors 

MANA 

JDAES 

SEAS 

6 

Ability to Simulate a D3SOE 

3 

2 

5 

5 

Support Availability 

5 

4 

2 

5 

Cluster Access 

5 

4 

2 

5 

Ease of Use 

5 

2 

3 

4 

Rapid Development 

4 

2 

2 

4 

Existing Models 

5 

2 

2 

4 

Ease of Analysis 

4 

3 

3 

4 

Output Data Density 

4 

3 

4 

3 

Learning Curve 

4 

2 

4 

3 

Behavior Monitoring 

4 

2 

3 

3 

DOE Tools Available 

5 

3 

2 

2 

Ease of DOE Execution 

5 

2 

2 

Total Value 

210 

127 

140 


As shown in Table 77, my qualitative assessment recommends the use of MANA 
for my dissertation research. MANA dominates the other two M&S packages, scoring 70 
points higher than the SEAS, the next best package. While SEAS is by far the most 
capable of modeling a D3SOE, its lack of available models and uncertainty regarding the 
modeling of ground combat makes its use questionable at best. Couple this with the fact 
that MANA beats or ties SEAS in every other area considered, many of which dealt with 
model usability in context of my research, and it is easy to justify the selection of MANA 
over SEAS and JDAES. Therefore, based on this analysis, I will be using MANA as my 
M&S package for this research. 

G. CONCLUSION 

The purpose of this chapter was to address the first of the supporting research 
questions as outlined in Chapter I, specifically, to identify what M&S packages where 
available that could accurately model the effects of a D3SOE on combat operations. In 
this Chapter I identified 14 potential M&S packages that after initial inspection seemed 
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as potential candidates for use in my work. After conducting a more thoughtful and 
detailed investigation of these alternative M&S packages, using primary and secondary 
screening considerations, I was able to reduce this list from 14 to just three. Following 
this, I conducted a detailed exploration of each of these three M&S packages to assess the 
appropriateness of each M&S package for use in my work, providing a solid foundation 
of analysis to justify the selection of MANA as the M&S tool for use in my research. 
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APPENDIX B. SPACE THREATS AND DEPENDENCIES 


At no prior point in the history of the U.S. Army have force multipliers like 
space-based capabilities been so heavily leveraged to mitigate operational risk. The Army 
uses its access to advanced space-based technologies and capabilities to secure and widen 
the tactical advantage over its adversaries in order to overpower and defeat them. By 
leveraging these advantages, the Army is able to maintain a smaller, more agile force that 
is capable of fighting and winning in a complex world, often in situations where it is out- 
manned, out-gunned, and out-resourced. Yet, Army requirements are expected to grow 
even in the midst of a significant reduction in force structure, and as the proverbial saying 
goes, the Army is now expected to “do more with lessT As outlined throughout the new 
AOC (U.S. Army 2014b), the future force will need to be smaller, lighter, more agile, 
more deployable, more capable, have a smaller footprint, have reduced support and 
sustainment requirements, and rely more on reach back support and capabilities, all the 
while maximizing its combat effectiveness to achieve overmatch with less. But this 
proverbial tightening of the belt does not come without risk. 

A. U.S. SPACE DEPENDENCIES 

Today, the space domain is truly a global commons, the ultimate high ground 
from which state and non-state actors alike will leverage to meet their own ends. While 
this realization is evident in U.S. national policy, the DOD has been slow to adapt to the 
change in the OE. While the United States overall market share of operational satellites 
has been falling over the last 10 years as more and more countries expand into the space 
domain, the United States remains heavily invested in the advantages that space affords 
both militarily as well as commercially. A summary of the current operational satellites 
can be seen in Table 78. 
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Table 78. Satellite Quick Facts through August 31 2015. 

Source: Union of Concerned Scientists (2015). 


Total number of operating .satellite.s: 1,.^05 

United States; .S49 

Russia: I.tl 

China: 142 

Other: 48.^ 

LEO: 696 

MEO: 87 

Elliptical: 41 

GEO: 481 

Total number of U.S. .satellites: .S49 

Civil: 21 

Commercial: 2.SO 

Government: 126 

Military: 152 


As shown in Table 78, even using the simplest of metrics, the United States is 
four times more invested in space than what many would consider our near-peer 
competitors. While this may or may not denote a U.S. dependency, it is easy to see how it 
could be interpreted as one. Even when considering just the 278 military/government 
satellites, a fairly conservative estimate, we see that they account for more than the total 
number of satellites operated by both China and Russia combined (273). Thus, from a 
purely quantitative perspective, U.S. Militarily relevant space assets account for over 
20% of all operating satellites worldwide; this is a huge percentage when considering the 
breadth of all space-based activities worldwide. Table 79 shows a consolidated table of 
the five Space Force Enhancement Mission Areas and their associate uses with regard to 
the six Army Warfighting Functions. This table was created to capture warfighter space 
dependencies, and while it is not all inclusive, it gives some needed detail on how the 
Army depends on space-based capabilities to support combat operations. 
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Table 79. Space to Warfighting Function Crosswalk. 

Source: DOA (2014, 3-20). 



Mission 

Command 

•Movement 
and Maneuver 

Intelligence' 

Fires 

.Sustainment 

Protection 

SMcc Force Enhancement Mission Area 

ISK 

■Geospatial 

Info 

* Situational 

awareness 

• Imagery 
•Terrain 

•Geospatial 

Info 

•Situational 

awareness 

•Imagery 

•Terrain 

•Geospatial, 
measures, 
and signal 
intelligence 
•Terrain 

• Battle 
damage 
assessment 

•Terrain 

Not 

applicable 

• Geospatial 

Info 

■Situational 

awareness 

Imagery 

•Terrain 

Missile 

Warninj’ 

■ Missile launch 
and impact 
‘Ballistic 
missile 
warning 

• Predicted 
Impact points 

•OPIR 
* Operational 
area 

awareness 

• Missile 
launch cueing 

Not 

applicable 

■ Predicted 
impact 
‘Ballistic 
missile 
warning 

l-ln^lronnu'nlai 

Monilorinu 

• Weather 

•Operational 

planning 

• Imagery 

• Mobility 

• Weather 

•Operational 

planning 

• Imagery 

• Mobility 

•Operational 

planning 

Not 

applicable 

.SATCOM 

• lil.OS comms 
•NI.OS comms 

• Rcachback 

• UI.OS comms 
•NI.OS comms 

• Rcachback 

• Reach 
forward 

• BI.OS comms 
•NI.OS comms 

> BLOS comms 
•NI.OS comms 

• BLOS comms 

• NLOS comms 

Position, 
nasication, 
and timing 

•FFT 

• JIK'P with KFT 
•Critical 
timing 

•FFT 

•JIK'P with FFT 
• Position, 
navigation, 
and timing 

•FFT 

•JIK'P with 
FFT 

• Precision 
targeting 
•NLOS Fires 
•JIK'P with 

FFT 

•FFT 

•JIK'P with 

FFT 

• In-transit 
Visibility 

•FFT 

•JIK'P with 

FFT 

1. S|>acc operations enable certain capabilities within the Intelligence warfighting function operations, but Space 
operations arc also dependent upon Army Intelligence to provide reconnaissance, surscillancc. and intelligence 
products and information. 

ni.OS beyond line of sight FKT friendly force tracking JIK'P joint battle command platform 

NLOS non-line of sight OPIK overhead persistent infrared SATCOM satellite communications 

ISK intelligence, surscillancc. and reconnais.sance 


As shown in Table 79, each of the Warfighting Functions listed across the top are 
saturated with dependencies from the Space Force Enhancement Areas listed on the side. 
To provide more detail regarding the contributions of the Space Force Enhancement 
Mission Areas to the Warfighting Eunctions, a brief description is provided: 

• The Mission Command function requires space capabilities to support SA 
and command and control. 

• The Movement and Maneuver function requires space capabilities to 
support planning and decision making for current and future operations. 

• The Intelligence function uses space capabilities to support building a 
more accurate assessment of the enemy and the OE. 

• The Eires function relies on space for targeting, information sharing, and 
precision fires. 
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• The Sustainment function uses space capabilities to support the speed and 
efficiency of logistical operations in support of combat, specifically 
friendly force tracking and communications. 

• The Protection function relies on space-based capabilities to provide SA of 
friendly and adversary forces, as well as early warning of impending 
attacks in order to give U.S. forces time to protect soldiers and systems. 

Each of these space force enhancement areas supports U.S. operations in one form 
or another. “Satellite communication (SATCOM) allows forces to operate over extended 
ranges; the Global Positioning System (GPS) delivers precise positioning, navigation, and 
timing; and space-based Intelligence, Surveillance, and Reconnaissance (ISR) systems 
provide unprecedented situational awareness of adversaries” (U.S. Army 2014a, 1). 
These three areas, plus missile warning and environmental monitoring will now be 
discussed in more detail. 

1. SATCOM 

Speed of communication as well as the quality and accuracy of information has 
always been a crucial factor in determining the outcomes of conflict. Today, this fact 
holds true, and SATCOM is the most operationally relevant space-based capability 
utilized by the DOD. By providing warfighters with “a global network of joint military 
and commercial communication satellites, operating forces at all levels of command to 
overcome limited infrastructure, execute reachback operations, enable two-way flow of 
data to critical nodes, provide support to special users, and increase overall command and 
control effectiveness” (DOA 2014, 3-10). Fundamentally, all space-based capabilities 
and services provided to the warfighter depend on communications. ISR, MW, PNT, and 
all other space-based capabilities utilize communications to deliver their services to the 
users, thus, SATCOM communications is the central cog in the Army’s informational 
war machine. Communications, specifically the ability to rapidly communicate 
information anywhere at any time, is the key driver of U.S. Military tactical advantage. In 
its simplest form, it is focused on providing users a better understanding of the OF, and in 
turn, allows leaders to make better and faster decisions, enabling the military to “out 
decide” its adversaries and gain tactical advantage. The ability to rapidly disseminate 
information, SA, and BMC2, gives the U.S. Army an unparalleled ability to quickly 
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execute the decision making cycle — a significant advantage over adversaries who require 
more time to gather, process, and disseminate the information needed to make decisions. 
Without SATCOM, the USGs ability to achieve and maintain overmatch will be 
significantly hampered, leaving the Army unable to provide many of the key space 
enabling technologies to the forces who can best exploit them. Generally, SATCOM can 
be broken down into four primary types. 

a. Narrowband SATCOM: UHF 

The U.S. military uses Narrowband SATCOM systems primarily in support of 
tactical operations, which provides highly mobile forces with easy to use, small, and light 
weight voice and data communication systems. Narrow band systems like UFO and 
MUOS “support secure voice and data transmission at relatively low data rates for mobile 
and fixed users” (DOA 2014, 3-12). The Army depends on narrowband SATCOM in 
environments where no infrastructure exists to support larger and higher capacity 
systems, like we expect to see in the future. Thus, according to the new AOC, the Army 
is expected to leverage these types of communication systems even more as the Army 
transitions into a more expeditionary force. Unfortunately, due to the spectrum 
characteristics that make systems operating at this band useful to tactical users - such as 
low power, small size, and rapidly deployable — systems at this frequency band are fairly 
vulnerable to adversary counter-space activities. 

Narrowband systems are extremely susceptible to nuclear scintillation and EMP, 
jamming, interference, as well as the space environment. Thus, operations in this band 
are significantly more risky than at higher bands. Additionally, because of the limit width 
of the band (from 0.3 to 3 GHz) as well as the large number of users, it is a crowded 
spectrum, and often subject to prioritization of bandwidth. Luckily, the relatively low 
data rate of the systems, the number of transponders available, the use of spot beams, and 
the U.S. military’s ability to dynamically re-allocate bandwidth mitigates the risk 
somewhat. Adversaries will be forced to weigh the cost of attacking these systems with 
the expected operational impacts. Those who choose to use counter-space capabilities 
will need to do so purposefully and will need significant resources of intelligence, time. 
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and continuous monitoring to accurately select the frequencies/transponders that they 
believe to be the most critical to U.S. operations. 

b. Wideband SATCOM: SHF/EHF 

The U.S. uses Wideband SATCOM systems in support of the operational level of 
military operations, primarily at the BDE and above level. These semi-mobile elements 
are less constrained by power, weight, and size limitations and require significantly more 
information and data capacity to conduct effective operations compared to the lower level 
tactical elements. Wideband SATCOM provides more capacity for users than 
Narrowband through greater bandwidth and additional channels, as well as 
“multichannel, secure voice, and high rate data communications for mission command, 
crisis management, and intelligence data transfer services” (DOA 2014, 3-12). The Army 
depends on Wideband SATCOM to provide the majority of its data requirements at the 
tactical and operational levels of war. As the informational demands of the warfighter 
continued to increase, the inability of existing systems to meet the need drove the 
development of high capacity communications systems. These systems, like DSCS and 
WGS, comprise the majority of the military communications architecture. Unfortunately, 
while these systems are less vulnerable to nuclear scintillation and have greater imbedded 
protection measures, they are still fairly susceptible to adversary counter-space activities. 

Wideband satellites provide the majority of voice and data requirements for 
operational warfighters when compared to both Narrowband and Protected SATCOM. 
Couple this dependency with the relatively small number of military Wideband systems, 
and the risk is even more pronounced, presenting a potentially lucrative target set for 
adversary counter-space capabilities. Fortunately these risks are mitigated in a few ways: 
the orbits of the systems; the width of the band (from 3 to 30 GHz); the use of spot 
beams; advanced modulation schemes; and dynamic frequency allocation. The 
combination of Wideband SATCOMs significance to U.S. operations and its expected 
vulnerabilities would likely draw significant attention from adversaries. Adversaries have 
the capability to limit the effectiveness of these systems, and while Protected and 
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Commercial SATCOM systems could be utilized to mitigate this capability loss, the 
timing, saturation, and persistence of an attack could steal the initiative from U.S. forces. 

c. Protected SATCOM: Narrow beam/spread spectrum SHF/EHF 

The U.S. uses Protected SATCOM systems in support of the operational and 
strategic levels of operations, primarily at the senior command level of military and 
government. These elements and organizations typically occupy fixed sites, with few 
limitations on power, weight, or size, and require some level of assured voice and data 
communications. Protected communications are typically reserved for the most vital 
requirements of the USG, that typically require “survivable voice and data 
communications not normally found on other systems” (DOA 2014, 3-12). Through the 
combination of modulation and protection schemes, protected communications are 
resistant to scintillation, interference, and jamming. While lacking the capacity and 
throughput of Wideband systems. Protected SATCOM provides users assured 
communication of the most critical national and strategic information. These systems, 
like MILSTAR and Advanced EHF comprise the backbone of the military 
communications architecture, where the majority of all strategic BMC2 is conducted. 

Unfortunately, while these systems are extremely difficult to attack, being nearly 
invulnerable to nuclear scintillation or jamming, they are still susceptible to adversary 
kinetic counter-space capabilities. While this risk is somewhat mitigated by the orbits of 
the systems, the combination of Protected SATCOMs’ significance to U.S. operations 
and their limited number will likely make them a top priority target for adversary kinetic 
anti-satellite weapons. While Wideband and Commercial SATCOM systems could be 
utilized to mitigate this capability loss, attacks against protected systems would have 
detrimental and cascading effects on U.S. operations at the strategic level. Without its 
eyes and ears, the U.S. may not have a full understanding of the OE, and thus may 
inadvertently make decisions that could significantly increase the chances of conflict 
escalation. Therefore, rational adversaries would likely give some strategic pause before 
attacking these systems, as Wideband satellites would likely have far more tactical 
relevance in a regional conflict without risking the chance of conflict escalation. 
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d. Commercial SATCOM 

The military needs information in order to drive its operational battle rhythm. The 
speed, quantity, and quality of information directly impacts the pace and outcome of 
battle, and gives the U.S. a tactical advantage when its use is maximized in coordination 
with operations. Recognizing this, military forces have developed an insatiable appetite 
for more and more information, yet this need for information has far outpaced the USs 
ability to provide it. This has forced the government to outsource much of its steady-state 
communications requirements to commercial SATCOM providers, who can offer 
“greater capacity that can be exploited to meet and augment the Army’s rapidly growing 
information needs” (DOA 2014, 3-13). Recent reports suggest that over 80% of the 
military’s steady state Wideband STACOM requirements are currently being met through 
the leasing of commercial bandwidth. While there are some potential security risks with 
this, the benefits are significant when considering adversary counter-space capabilities. 

Mitigating the potential impacts to combat effectiveness of U.S. forces can be 
achieved in a few ways. First, there are a lot of commercial SATCOM providers, and 
attacking one, or even a few, would not likely have a large enough impact to the opposing 
force to justify the effort. Second, commercial SATCOM systems are not government 
satellites, so they are not technically military targets, so there would be some 
international law and policy factors to consider. Thirdly, commercial SATCOM providers 
support numerous countries and clients, and an attack on a SATCOM system will likely 
impact many other countries than just the one targeted. This is likely counterproductive to 
the aims of the adversary, who will likely want to keep the conflict small and more 
manageable. Thus, U.S. leveraging of commercial SATCOM presents adversaries with a 
relatively difficult problem to solve. 

2. ISR 

The U.S. Army is a consummate consumer of ISR. As an organization that 
executes current and future planning cycles continuously, the Army has a high demand 
for current and relatively high resolution Electro Optical (EO), Infrared (IR), and radar 
imagery in support of planning. While ISR can be collected from numerous sources. 
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including Air, UAV, High altitude, Near Space, and Space, space-based platforms are the 
only dependable and routinely available source of high resolution imaging, and are “a 
crucial enabler supporting all Army operations” (DOA 2014, 3-6), providing the U.S. 
nearly unrestricted access to denied areas. The information provided by ISR is often 
critical to decision making, and “supports the development of intelligence that supports 
mission success, and other actions that may influence the commander’s current and future 
operational decisions” (DOA 2014, 3-6). Without ISR, the USGs ability to maintain up 
to date SA regarding adversary force distribution, disposition, strength, and location will 
be severely compromised. This will force commanders to slow the pace of battle as they 
seek other less efficient means to fill the informational gaps left by the loss of ISR. Thus, 
the rate in which the Army can execute its decision making cycle is slowed, reducing its 
operational overmatch and giving the adversary more time and space on the battlefield. 

The greatest risk to collection assets is to the high resolution U.S. space-based 
ISR assets because of there are a relatively small number of them. Couple this fact with a 
high demand signal for the capabilities, and one can quickly see how adversary actions 
against even a single system can have significant impacts on operational effectiveness. 
The relatively low mission altitudes and advanced capabilities of these systems leave 
them extremely vulnerable to attacks, especially from laser and kinetic attacks. Thus, a 
relatively small adversarial investment into counter-space capabilities like lasers, focused 
on temporarily blinding U.S. ISR assets, has the potential for a huge return on 
investment, making them attractive options for future adversary R&D programs. While 
the risk from laser dazzling attacks are high, they are relatively short lived and their long 
term impacts to operations are low to moderate depending on the timing of the attack. On 
the other hand, while the risks from kinetic attacks are lower, the impacts are far more 
significant. Not only do successful kinetic attacks destroy target satellites, they have the 
potential of creating massive debris fields that can significantly hamper other satellite 
operations and possibly even destroy them. 

Luckily the laser and kinetic threats are mitigated in a few ways. First, only three 
countries — Russia, China, and the U.S. — have demonstrated the capability to conduct 
kinetic attacks on space systems’, thus there is a relatively small number of these 
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weapons. Second, the destruction of a U.S. satellite could be considered an act of war, 
and thus, it is a red line that would give most adversaries pause before executing. Lastly, 
by using commercial imagery to supplement U.S. collection shortfalls, which currently 
provides greater than 50% of Army imaging requirements, the United States has 
disaggregated its “eggs” to other baskets, giving adversaries a larger number of potential 
targets that make it more difficult to effectively degrade U.S. collection capability. 

3. PNT 

The ability to provide the warfighter at all levels of operations with detailed 
position and timing data, anywhere in the world, has given the U.S. military a 
significantly better understanding of the OE. The Army depends on GPS to support 
offensive and defensive operations, which are “enabled through precision navigation aids 
and through networked mission command, control, and communications capabilities that 
depend on timing signals from the GPS transmission” (DOA 2014, 3-15). The Army uses 
GPS to reduce uncertainty, both in adversary as well as friendly activities. More accurate 
location accuracy leads to more accurate SA, more effective maneuver, and more 
accurate targeting. More accurate timing data leads to more efficient and effective 
networks and communications. This allows decision makers to more accurately assess 
operations, the threat, and the environment, and allows them to make quicker and better 
informed decisions, which can drastically increase the operational tempo of the Army. 
This increased pace of battle gives the Army a tactical advantage over adversaries who 
are unable to execute their decision making cycle as quickly. Thus, it is easy to see how 
the loss of PNT can significantly impact the Army’s ability to execute operations 
effectively. Any degradation to PNT will induce uncertainty into the forces SA, and thus, 
give decision makers pause as they access the risk of that uncertainty. 

Without GPS, the USGs ability to achieve and maintain operational overmatch 
could be significantly hampered, especially at the tactical level where the advantages of 
GPS are concentrated in order to generate RCP over opposing forces. Loss or even 
degradation of the GPS signal will induce some level of uncertainty into the decision 
making process, which will inevitably lead to a slower decision making process, and thus, 
a slower pace of battle. Typically, this will favor the defense, and thus, will likely be 
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detrimental to U.S. operations, that rely on a high operational tempo to achieve 
operational overmatch by overwhelming adversaries quickly. The U.S. is not designed or 
optimized to execute slow or prolonged operations, and any impact that causes a decrease 
in operation tempo will have a corresponding impact on combat effectiveness, likely seen 
as a higher number of casualties, and increased length of battle. Luckily, the threat to 
GPS is a fairly localized problem when considering combat operations, where the threat 
is aimed more towards impacting users than the system themselves. 

Mitigating the risk to U.S. combat effectiveness due to adversary actions targeting 
GPS can and are achieved in a few ways. First, the risk is somewhat mitigated by the 
orbits of the systems themselves, because the altitude of MEO poses a significant 
targeting problem for both kinetic and non-kinetic weapons. Second, there are a large 
number of GPS satellites, and attacking one, or even a few, would not likely have a large 
enough impact to the opposing force to justify the effort. Third, GPS satellites are now 
considered civil satellites, and fall in to a “gray area” when considering targeting, so there 
would be some international law and treaty factors to consider. Forth, GPS is primarily a 
civil service, providing services to users worldwide, and an attack on a GPS satellite 
would likely impact many other countries and users, and cause an international uproar, 
which is likely counterproductive to the aims of the adversary. Lastly, the U.S. is 
reducing its dependency on GPS alone, and increasing its use of other PNT systems like 
GLONASS, Galileo, IRNSS, and BeiDou. Because of these factors, it is unlikely that an 
adversary would attack a GPS satellite. More likely, adversary forces would seek to 
attack local military users of the GPS signal through the use of local jammers to deny and 
degrade U.S. positional knowledge at a tactical level. As a result, the adversary could 
achieve many of the same objectives as attacking many GPS satellites, at least locally, 
but without the difficulty or political backlash of attacking a space-based system. 

4. Environmental Monitoring 

Environmental monitoring is a fairly broad space mission area that accounts for 
roughly 20% of the U.S. governments overall earth observation activities and 
incorporates everything from SAR, disaster relief, weather monitoring, and scientific 
earth sensing missions. Of these, weather monitoring is the most significant when 

351 



considering combat operations, especially when considering the impacts that weather can 
have on operational measures of effectiveness. Consequently, weather forecasting is an 
important step in the intelligence preparation of the battle field prior to mission 
execution. During this step, two types of weather must be considered with respect to their 
potential operational impacts. The first is terrestrial weather, where by the analysis of 
“weather data, identifying potential weather effects, and assessing the impact of weather 
on systems, tactics, and operations provide vital information for commanders to 
optimally employ their forces” (DOA 2014, 5-3). The second is space weather, which 
analyzes the effects of solar activity and its impacts on the function and performance of 
space-based systems. Space weather is unpredictable in both occurrence and scale — thus, 
“space weather events may adversely affect PNT, surveillance and reconnaissance 
missions, as well as terrestrial- and space-based SATCOM capabilities” (DOA 2014, 3- 
7). This is a relatively new area which has grown in importance as our dependencies on 
space have increased, driving a need to monitor the space environment in order to 
anticipate, mitigate, and recover from the impacts of the space environment. While space 
systems are designed to withstand the space environment, our ability to monitor, 
understand, and anticipate space weather events is relatively limited, and thus, while the 
risk is low, the overall impact can be significant. 

Like missile warning, the overall risk to environmental monitoring systems from 
adversary counter-space capabilities is fairly low. This is due to the relatively benign and 
passive nature of these systems, the large number of them, and the relatively open and 
free access to the information they provide. Most environmental monitoring systems 
share their information through a worldwide partnership in which the USG participates. 
This means that USG environmental monitoring systems provide data to all nations, and 
vice versa, we have access to every other participating country’s data. Thus, attacks 
against environmental monitoring systems are fairly meaningless, and makes “them 
relatively low-payoff targets with a significant degree of political risk for the attacker” 
(Heginbotham et al. 2015, 254). When considering the relatively small number of 
counter-space weapons capable of affecting these systems, as well as the relatively low 
payoff for doing so, it becomes clear that it is unlikely that these systems will ever be 
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attacked. Because of this, as well as the relatively small percentage of environmental 
monitoring systems when compared to the overall number of U.S. satellites, just 4%, they 
will not be addressed in detail in this dissertation. 

5. Missile Warning 

Missile warning accounts for roughly 6% of the U.S. governments’ overall earth 
observation activities. While not a mission area which I will address in detail during this 
dissertation, space-based missile warning does play an important role in combat 
operations. Specifically, space-based missile warning has two primary contributions to 
the overall combat effectiveness of supported forces. First, it provides vital and timely 
information on adversary ballistic missile launches into the Joint Integrated Air Defense 
network much earlier than terrestrial systems, queuing positioned air defense assets and 
providing the detailed information needed to execute intercepts. Second, it provides key 
information to ground forces on potential target areas minutes ahead of the expected 
impact, giving ground forces the time needed to execute protection measures to minimize 
damage and casualties. Thus, space-based missile warning plays a key role in force 
protection by providing commanders with “early warning of enemy ballistic missile 
launches via the TES reporting” (DOA 2014, 3-9). In fact, the space-based missile 
warning network is the United States’ primary strategic early warning system for 
detecting ICBM launches, an essential enabler of U.S. national security. 

Luckily, the overall risk to these systems from adversary counter-space 
capabilities is fairly low because of three key mitigation factors. First, the U.S. has 
inferred in statements and in policy that any attack on this network could be perceived as 
a precursor to nuclear attack and that the USG would consider first use of nuclear 
weapons as a potential response. This strategic message draws a clear red line that would 
give most rational adversaries pause before crossing it. Second, the majority of these 
systems are located in GEO and HEO, and by nature of these extremely high orbits 
makes it difficult to attack them, with only a few adversaries even capable of doing so. 
Lastly, the overall impact to combat operations for the U.S. is fairly low when compared 
to other systems like SATCOM and ISR. Together, “these characteristics would 
complicate Chinese efforts to dazzle, or blind, the system with lasers, thereby reducing 
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the risk of attack” (Heginbotham et al. 2015, 256). Thus, adversaries are not likely to 
waste limited counter-space assets on these systems when there are higher payoff targets 
available, even though the U.S. has relatively few of these systems. 

B. THE THREAT 

If history has taught us anything it is the fact the humans have never developed a 
weapon and not used it. Our own doctrine states that “Instead of attacking enemy 
strength, the goal is the application of our strength against selected enemy weakness in 
order to maximize advantage” (U.S. Marine Corps 1997, 37), so why would we not 
expect the adversary to do the same? As a result, knowing your adversary and their 
potential capabilities becomes critical to understanding the risk. Likewise, without an 
understanding of our own vulnerabilities, it would be difficult to assess how an adversary 
might attack, and how that attack could affect us. It is now more important than ever to 
maintain an accurate assessment of adversary counter-space capabilities as well as U.S. 
vulnerabilities in order to accurately assess and prepare for the inevitability of operations 
in a D3SOE. As Tellis (2014) states, “Chinese strategists are by necessity drawn to the 
idea of attempting to neutralize American space capabilities. This lure becomes all the 
more tantalizing because not only is U.S. space superiority critical for the success of 
American military operations but its space architecture is as a rule remarkably vulnerable 
to offensive actions undertaken by an adversary” (4). Thus, the Chinese will likely seek 
to target these vulnerabilities, by expanding efforts to exploit out perceived weaknesses. 

This growth is best demonstrated by the Chinese, who’s development of counter¬ 
space weapons has mostly gone un-checked over the last 15 years as the United States 
focused on its Wars in the middle-east, and because of this, China was able to rapidly 
build and field an impressive array of counter-space capabilities that currently surpass 
any other country in the world. The Chinese recognized their friction with the United 
States, which has been highlighted in recent documents which admit that “Chinese 
defense planners are deeply consumed by the necessity of preparing for an armed 
confrontation with the United States, which they clearly recognize as a superior military 
power” (Tellis 2014, 3). China sees conflict with the U.S. as an inevitable part of their 
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destiny, and have been preparing for it for years. Most PLA literature focuses on this 
inevitability, and highlights the importance of “destroying, damaging, and interfering 
with the enemy’s reconnaissance ... and communications satellites” when considering 
conflict with the United States (DOD 2014a, 32). Yet recognizing the advantage of 
attacking U.S. satellites and having the capability to do so is one thing, but having the 
willingness is another. Unfortunately, based on additional PRC writings, it seems that the 
PRC is willing and able here as well, stating that “prosecuting counterspace operations in 
a crisis may be rational for China in any significant Sino-U.S. conflict along its 
periphery, even though Beijing itself stands to lose considerably as a result of the 
expected American riposte” (Tellis 2014, 3). 

To give some context regarding the potential threat, I have provided some 
additional insight to a list of major Chinese counter-space development activities 
mentioned in the testimony of Ashley J. Tellis (2014). 

• Direct-Ascent (DA) and Co-Orbital (CO) ASATs. The development of these 
weapons offer the PRC the means which to destroy with high likelihood adversary 
satellites. Additionally, they also act as a deterrence capability that can be 
leveraged to achieve political objectives. 

• Electronic Warfare (EW) SATCOM/GPS jammers. These weapons allow the 
PRC to impact key U.S. capabilities and dependencies with non-lethal effects. 
These are difficult to detect and attribute, and cause U.S. forces to react to 
mitigate the impact, which takes time and resources, slowing the pace of battle, a 
key force multiplier for the U.S. 

• High- and low-energy lasers and high-powered microwave weapons. These 
weapons offer the PRC the capability to achieve a range of impacts on targeted 
satellites, from disruption and degradation, to destruction. This gives the PRC 
various attack options to achieve the desired effect which are difficult to detect 
and attribute, as well as difficult to consider attacks that are escalatory in nature. 

• Space-Object Surveillance and Identification systems (SOSI). By expanding and 
improving its SOSI network, the PRC will be capable of more accurately tracking 
and targeting space assets, a key aspect of SSA. 

• Computer Network Attacks (CNA). Space systems are just as vulnerable to 
network attacks as any other network. Because of the relative isolation of on-orbit 
systems, CNA is typically the easiest method in which to attack space systems 
and their ground networks. 
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1 . 


Threat Areas 


Thus, the PRC seems to be putting substantial effort into a range of development 
activities aimed to counter U.S. capabilities and dependencies on space-based systems. 
Of these, four are of immediate concern to me, and will be discussed in more detail. 

a. GPS Jamming 

In many ways, GPS jammers can be considered the entry level or “first tier” 
counter-space capability. Most nations and many non-state actors have acquired GPS 
jammers and actively use them in military operations. These systems are readily 
available, easily procured, and easily manufactured with relatively little experience or 
knowledge needed. In fact, instructions for building GPS jammers can be found on 
YouTube. Now the effectiveness of these homemade jammers is debatable, it serves to 
highlight that almost all state and non-state actors have some level of counter-space 
capabilities. Of more concern to the U.S. are the military grade GPS jammers, which can 
be purchased from various manufactures, to include companies in Russia, China, India, 
and many more. As an example, one marketed hand-held Russian jammer “can deny 
access to GPS out to 50 miles; a slightly larger version can jam up to 120 miles” (Garino 
and Gibson 2009, 276). While the effectiveness of such jammers depends on many 
factors, the weak signal of GPS makes it susceptible to jamming, especially in close 
proximity to jammers. Thus, it does not take allot of effort to degrade the GPS signal. 

The impacts from GPS jamming are fairly specialized and depend in many ways 
on the level of operations. Army forces at the tactical level are typically the least effected 
by GPS jamming because they are operating on the terrain and typically have some 
knowledge of the battle space. Army accuracy requirements at this level of operations are 
typically less restrictive, which significantly reduces the dependency on PNT. For 
example, direct fire forces like infantry, armor, and mechanized infantry can physically 
see the adversary they are fighting, thus they are relatively unaffected by GPS jamming 
once in combat. Though loss of GPS would likely have some negative impact on the 
speed of maneuver, and slow force movement due to positional uncertainty and the 
resulting delays in BMC2, it can pretty much be ignored during combat. When looking at 
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the overall context of the campaign, where large periods of non-combat are included, 
than we can expect to see more effects from GPS degradation on measures of 
effectiveness like length of battle and number of casualties. Indirect fire systems like 
Artillery will be more impacted, especially when using GPS guided munitions. These 
impacts can be partially mitigated by using non-GPS aided munitions and forward 
observers to direct fire, but this will induce error, increase the number of salvos, decrease 
effectiveness, and decrease the pace of battle: all of which will impact overall combat 
effectiveness. Aviation, intelligence, and logistics forces will be even more impacted 
because these forces are dependent on PNT for efficient execution of their mission sets. 
For other organizations the problem faced by GPS jamming is even more significant. 

Other organizations like the Air Force and Navy will be significantly impacted by 
GPS jammers, especially in the context of their contribution of combat power in support 
of the ground force. Because of the characteristics of signal propagation, GPS jamming 
range increases with altitude. Thus, for higher altitude system like Air Force Bombers, 
and long range fires like Navy cruise missiles, the GPS jamming range can be significant. 
Couple this impact with the dependency of modern aircraft, weapons, and other deep 
strike systems on GPS, and we have a situation where GPS jamming can significantly 
limit the effectiveness of the strategic deep strike capabilities of the US. Unfortunately, 
the military has become dependent on deep strike capabilities to support forward 
deployed forces, and this dependency is expected to grow as the future force downsizes. 
As the new AOC states, the Army will depend on more of these capabilities to provide 
the necessary combat power to achieve overmatch. If the effectiveness of deep strike 
capabilities can be mitigated through adversary use of GPS jammers, then the ability of 
forward deployed forces to generate overmatch is put at risk. But once again, there is 
currently no means to quantify this impact, thus no means to evaluate when that transition 
may occur, and thus, no way to effectively plan for it. 

b. SATCOM Jamming 

SATCOM jammers can be considered the “second tier” of counter-space 
capabilities. While SATCOM jammers are slightly more complex than GPS jammers, the 
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only significant technological advancement needed to develop and employ SATCOM 
jammers is the ability to track and target a specific communications satellite, and then, 
having the needed pointing accuracy of the transmitter, put enough power on the target to 
jam its uplink. None of these technologies is considered cutting edge, but the difficulty in 
integrating them into a single SoS does provide enough of an obstacle to preclude a large 
number of adversaries from pursuing these counter-space capabilities. Additionally, 
because the United States uses such a large amount of SATCOM, hosted by numerous 
providers on both commercial and military satellites, and using all bands from narrow to 
protected bands, it is difficult for adversaries to achieve success. In order to achieve the 
desired impacts to U.S. operations, an adversary would need to develop and field a large 
number of these systems and employ them simultaneously. 

The impacts from SATCOM jamming are again fairly dependent on the level of 
operations. Like with GPS, Army forces at the tactical level (BN and below) are typically 
the least affeeted by SATCOM jamming beeause they typically communicate with line- 
of-site HF communication systems, which do not rely on satellites. Thus, over short 
periods of time, SATCOM jamming has fairly limited impact on combat operations. But 
as we move above the Battalion level, and consider longer periods of conflict, the impact 
of SATCOM jamming on Army organizations becomes more apparent, especially at the 
operational and strategic level of war. 

At these levels, impacts to SATCOM can significantly restrict the flow of 
information, especially the BMC2 and SA information between higher and lower Army 
echelons that is necessary to maintain the high operational tempo of combat forces which 
has given the U.S. its tactical advantage. As SATCOM jamming increases in quantity and 
duration, U.S. forces should expect a steady degradation of combat effectiveness. This 
degradation is primarily due to a lack of information, which induces greater uncertainty 
with regard to the USs understanding of the OE, and can increase the time needed for the 
U.S. to exeeute its decision making process. Additionally, the ability of forward deployed 
ground forees to pass and request strategic fires and deep strike eapabilities may be 
hindered, which again will negatively impact combat effectiveness of ground forces who 
rely on this combat power to achieve and maintain operational overmatch. Thus, over 
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time, SATCOM jamming can potentially reduce the combat effectiveness of U.S. forces 
to a point in which the Army’s ability to win is put at risk. In other organizations, like the 
Air Force and Navy, the impact of SATCOM jamming is again much more significant. 

Unlike the other services, the Navy relies on SATCOM to provide almost all of its 
SA, BMC2, and intelligence needs. The Navy is always deployed, to areas where there is 
no terrestrial communication solution, and therefore, any degradation to SATCOM links 
due to adversary counter-space activities will have a negative impact on the effectiveness 
of Navy forces. Without SATCOM, Navy groups operating at sea will be forced to 
reduce their operational tempo due to a significant increase in uncertainty. Like we saw 
with ground forces operating in a degraded GPS environment, the Navy would be forced 
to slow its pace of battle to conduct the necessary reconnaissance activities needed to fill 
the gaps that would have otherwise been provided through SATCOM. Because of this, a 
larger portion of the Navy’s available combat power would need to be diverted to meet 
its own requirements, and thus, not be capable of supporting the ground force. This is yet 
another example of how counter-space activities can reduce the combat power 
contribution of supporting forces to the ground force, which relies on the Navy’s 
contribution to achieve operational overmatch. 

SATCOM jamming can impact the Air Force in a few ways, all with varying 
impacts on the Air Forces’ ability to support ground operations. War planes operating in 
support of combat operations will typically have HF comms with the units they are 
supporting and thus, are not as impacted by SATCOM jamming. However, the majority 
of the higher level Air Force operations are heavily dependent on SATCOM, especially at 
the operational and strategic level. The foundation of Air Force operations revolves 
around the daily execution of the Air Tasking Order (ATO). This planning process is 
extremely efficient, yet it is also heavily dependent on SATCOM to gather information 
and to disseminate orders. Almost the entire intelligence tasking and collection network 
relies on SATCOM to request and provide the desired intelligence products needed to 
execute the targeting process effectively, as well as to conduct the BDA following 
strikes. Degradation of SATCOM could impact the understanding of the OF, resulting in 
a less effective targeting and assessment process as well as a slower operational tempo. 

359 



Another significant impact to Air Force operations due to SATCOM jamming is the 
impact to its fleet of UAVs, which depends on SATCOM for their operational control. 
UAVs are an incredibly important capability that provides a significant contribution of 
combat power to the ground force by providing additional capability to collect 
intelligence, as well as to provide fires in support of combat operations. However, these 
deep strike capabilities can be highly sensitive to jamming, which can reduce the 
effectiveness of UAVs and potentially negate their contributions to combat power. 

National command and control will also be vulnerable to SATCOM jamming. 
While the majority of these communications would likely be conducted over protected 
communication bands, we know that the throughput of these systems is extremely 
limited, allowing only the most import and critical information to be passed. Thus, there 
will be a significant amount of information that can potentially be negated due to 
adversary SATCOM jamming. Because this information contributes to an operational 
commanders’ understanding of the OE, reduction in the throughput of these links can and 
will have an impact on metrics of operational effectiveness. Additionally, SATCOM 
jamming can also potentially impact the ability of the USG to execute its oversight and 
control of military operations over long distances. This reduced control can induce 
operational uncertainty, which may result in a slower decision making process. 

According to the new AOC, the Army of the future can expect to be rapidly 
deployed into areas where it will have little to no infrastructure support and likely be 
outmanned, out gunned, and out resourced. To achieve and maintain operational 
overmatch against these odds, the Army will be highly reliant on support from the Navy, 
Air Force, and National Strategic capabilities to provide the necessary combat power. If 
the effectiveness of Navy, Air Force, and national capabilities can be mitigated through 
adversary use of SATCOM jamming, then the ability of forward deployed forces to 
generate overmatch is put at risk. But once again, there is currently no means to quantify 
this impact, thus no means to evaluate when that transition may occur, and thus no way to 
effectively plan for it. 
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c. Direct Energy Weapons 

Direct Energy Weapons can be classified as advanced counter-space capabilities 
and occupy what I will call the “third tier” of counter-space capabilities. These are truly 
technologically advanced capabilities and prove to be significant obstacles to even the 
most determined adversary. Non-state actors and the majority of nation states will not 
have these capabilities, primarily due to the extreme complexity, cost, and resource 
requirements needed to develop and field such systems. Direct Energy Weapons are 
therefore reserved for just a handful of the largest and most technologically advanced 
countries and can be classified into three primary weapon types: lasers, RE, and Particle 
Beam. These all have varying levels of capabilities and potential impacts, with 
complexity ranging from advanced technology, as we see with lasers, to the nearly 
theoretical realm of Particle Beam Weapons. Of these three, lasers are the most common 
and currently the only direct energy weapon in operational use today. Consequently, I 
will only be considering lasers in this dissertation. 

Easers are dual use technologies that prove to be difficult to mitigate. Eow power 
lasers are routinely used for range finding, which is essential for maintaining a nation’s 
Space Situational Awareness (SSA). Thus, it is difficult to classify lasers as weapons 
until they are used as such, and even then it is often difficult to detect and attribute this to 
any specific adversary. Eow power lasers can easily be used to disrupt U.S. satellite 
imagery by “dazzling” our sensitive sensors. While laser dazzling is typically non¬ 
destructive, it does interfere with the United States ability to maintain its informational 
advantage, which can have negative impacts on combat operations. This problem is 
further complicated by the fact that laser counter-space capabilities can be increased by 
increasing the output power. Thus, through the technical expansion of current laser 
capabilities, an adversary could be capable of developing systems that are far more 
capable, with potential impacts including everything from disruption through dazzling, to 
degradation and potential destruction of targeted satellites. Euckily, the effects of lasers 
are generally restricted to satellites at low altitudes like EEO. Satellites at higher altitudes 
like MEO and GEO, and more protected due to the laser scattering and dissipation effects 
that reduce the effectiveness of the laser at greater ranges. 
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The difficulty in mitigating the potential impacts from lasers is threefold. First, it 
is difficult to classify reversible and/or temporary impacts as attacks, especially enough 
to justify a kinetic response. Second, it is often difficult to prove that an attack even took 
place, and even more difficult to attribute the attack to a specific adversary. Third, given 
that the first two are addressed, we often do not even possess the capability to attack the 
source. For example, the PRCs laser range finding “network consists of five fixed 
stations located at space observatories in Shanghai, Changchun, Beijing, Wuhan, and 
Kunming. At least two mobile systems are also available” (Heginbotham et al. 2015, 
246). So, even given a destructive attack on U.S. systems that we can positively attribute 
to an adversaries laser, what could be done about it? All of these systems are located in 
areas of mainland China that are extremely difficult to target, and the mobile systems 
would be nearly impossible to target. Obviously, we would need to reciprocate, but how? 
In the case of China, the lasers themselves are likely too difficult to attack, and thus, they 
would likely continue to impact U.S. systems. Thus, we find ourselves in a situation 
where we can expect to be continually impacted by these systems and should expect to 
operate with degraded collection capabilities in any conflict with the PRC. Even if an 
adversary chooses not to employ its lasers as weapons, they will “still be an important 
element in the counterspace ‘kill chain,’ providing data of sufficient precision to target 
U.S. satellites with other weapons” (Heginbotham et al. 2015, 247). Thus, these systems 
pose a threat in all roles and must be accounted for in operational planning. 

The potential impacts of adversary use of lasers on combat effectiveness are again 
fairly dependent on the level of operations. The primary target for non-destructive lasers 
will be ISR assets, and thus, the intent of adversaries will likely be to blind U.S. 
collection assets in an effort to mask adversary actions and to induce uncertainty into the 
U.S. decision making cycle. This is less effective the closer to the forward line of troops 
we get, and thus. Army forces at the tactical level (BN and below) are again the least 
affected. Tactical forces do not rely on current high resolution imagery as much as higher 
echelon planners do because they typically have local collection assets that maintain 
contact with the adversaries’ forces. But as we move above the Battalion level, and 
consider longer periods of conflict, the impact of adversary use of lasers can become 
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more apparent, especially at the operational and strategic level of war. Once we start to 
consider lasers at higher power levels, where they can damage/destroy U.S. assets, the 
potential impacts become much more severe. In this case, all satellites at LEO become 
potential targets, significantly complicating potential mitigation plans. 

Decision makers depend heavily on space-based ISR to support operational 
planning and lasers can significantly restrict the availability of these products. Luckily, 
due to the large number of potential commercial and government sources of ISR, it is 
unlikely that an adversary could ever completely negate the United States’ access to 
space-based collection. But even if this is true, adversaries can still negatively impact 
U.S. operations by reducing the availability of ISR. Doing so would induce operational 
delays as decision makers wait for key information, significantly slowing the United 
States’ operational tempo and thus, reducing tactical advantage. As adversary lasers 
increase in quantity, duration, and capabilities, friendly forces should expect a steady 
decrease in collection capacity and a degradation of combat effectiveness, especially over 
longer periods of conflict. This would again introduce uncertainty with regard to 
understanding of the OE and would increase the time needed for the Army to execute its 
decision making process. In other organizations, like the Air Eorce and Navy, the impact 
of laser weapons will be similar. Thus, over time, laser weapons could reduce the combat 
effectiveness of U.S. forces to the point where we put our ability to win at risk. 

d. ASAT Weapons 

Anti-Satellite weapons can be classified as extremely advanced counter-space 
capabilities, and occupy what I will call the “forth tier” of counter-space capabilities. 
These systems cannot be procured, and are extremely difficult to develop and 
manufacture, requiring advanced expertise in space, targeting and tracking, launch, 
ballistic missiles, intercept technologies, propulsion, and exo-atmospheric maneuver. The 
technological requirements needed to develop these systems are only possible within a 
few of the largest and most technologically advanced countries with well-established 
space programs and substantial resources in both time and money. ASAT weapons can be 
classified into three primary weapon types. Direct Ascent-Low Earth Orbit (DA-LEO), 


363 



Direct Ascent-Geostationary Orbit (DA-GEO), and Co-Orbital (CO). These all have 
varying levels of capabilities and potential impacts, as well as overall complexity. Of 
these three, DA-LEO ASAT weapons are the only ones which have been operationally 
tested, but all three have been demonstrated sufficiently enough to consider them 
operationally feasible. Because of the potential threat of these systems we must consider 
all of them as operational. We can expect advanced adversaries to maintain relatively 
small quantities of DA-ASAT weapons, as well as 1-2 prototype GEO and CO ASAT 
weapons. Thus, all three of these systems will be considered in this work. 

DA-ASATs (EEO) are the most well-known and established ASATs currently in 
operation. These systems are capable of being launched from the ground, ships, or 
aircraft, and intercepting a target satellite in EEO. The United States and Russia have 
both had this capability since the 1980s, with both countries testing them multiple times 
during the height of the cold war. These weapons fell out of developmental favor after 
both the United States and Russia agreed that such weapons were counter-productive to 
the safe and secure access to space by both parties. While the weapons were maintained 
in the inventory, all significant development of these systems was halted. It was not until 
the January 2007 launch of a Chinese DA-ASAT that the U.S. reinvigorated its counter¬ 
space R&D efforts. This launch exposed the threat to U.S. systems, and “proved that 
China can range critical U.S. space systems in low Earth orbit, such as meteorological 
and electro-optical surveillance satellites” (Tellis 2014, 1). It is expected that the PRC has 
a handful of these systems, and thus, for this work, we will assume that an adversary is 
capable of completely destroying any U.S. asset in EEO. While DA-ASATs (EEO) are 
technically the only type of ASAT weapons that can be considered operational, there are 
other less mature systems capable of impacting U.S. operations as well. 

DA-ASAT (GEO) weapons are a logical evolution of DA-ASAT (EEO) weapons. 
By increasing the weapons range an adversary is able to increase the number of potential 
targets, to include high value national systems typically found in MEO and GEO. By 
achieving the capability to attack satellites in GEO, an adversary can now put at risk all 
U.S. assets, regardless of their orbits. The May 2013 launch of an unknown PRC system 
was suspected to be a DA-ASAT (GEO) test, and if true, would prove that the PRC has 
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the capability “to place a kinetic kill vehicle on a trajectory to deep space that could reach 
medium earth orbit (MEO), highly elliptical orbit (HEO), and geostationary Earth orbit 
(GEO)” (Weeden 2014, 1). This is a significant capability, and one that the U.S. may or 
may not be capable of accounting for. Even though this capability has not been 
operationally tested or acknowledged, most planners now expect that the Chinese have 
this capability. Consequently, we must consider the potential impacts of this weapon on 
operations in future planning. While the PRC likely only has a few of these weapons, we 
must assume that they are capable of completely destroying any U.S. asset out to GEO. 

Co-Orbital ASAT (Co-ASAT) weapons more closely resemble satellites than 
weapons, and for good reason, a CO-ASAT is a satellite. The only difference is that a 
CO-ASAT was designed to impact satellites, and thus, be a weapon. Technically, any 
satellite that has the capability to maneuver could possibly be used as an ASAT. And 
though it requires some detailed knowledge of the space environment and the target, the 
technology itself is not nearly as complicated as DA-ASATs. Any country that has an 
active space program already has all the technology needed to produce and use these 
systems. And while they have significant legal and treaty implications, specifically 
revolving around the topic of the weaponizing space, the fact that such systems are 
difficult to discern from other satellites makes it extremely difficult to classify them as 
weapons. Because of the benefits and flexibility such systems provide, China has 
“embarked on a programme to develop a co-orbital anti-satellite interceptor, launched 
from Earth into a temporary parking orbit from which it then manoeuvers to attack its 
specific target” (Tellis 2007, 54). Again, even though this system has not been 
operationally demonstrated, we must assume that the capability exists, and thus, we must 
consider all potential effects that such a system can have on U.S. operations. 

2. Vulnerabilities 

Because “PEA writings emphasize the necessity of ‘destroying, damaging, and 
interfering with the enemy’s reconnaissance ... and communications satellites,’ 
suggesting that such systems, as well as navigation and early warning satellites, could be 
among the targets of attacks designed to ‘blind and deafen the enemy’” (DOD 2014a, 32), 
the United States must take a more proactive role in mitigating the risk it faces when 
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considering the uncertainty of future combat environments. Yet, the focus of these efforts 
should align with the potential threats, which as discussed, are based on the perceived 
vulnerabilities. Thus, I will focus the majority of analysis addressing the potential 
impacts on U.S. SATCOM, ISR, and PNT. 

a. U.S. SATCOM 

As adversaries develop more and better SATCOM jamming capabilities, we can 
expect the access to critical information links to be put at greater risk. Couple this with 
the increase dependency on SATCOM by U.S. forces as we transition under the new 
AOC, as well as the proliferation of SATCOM jamming technologies to new and 
emerging adversaries, and this threat is only compounded. What this means for the 
warfighter is that as jamming increases, the overall availability and flow of information 
will decrease for most users due to the re-allocation and re-prioritization of available 
bandwidth. As the jamming surpasses the availability of redundant communications, 
lower priority users will begin to see a significant portion of their bandwidth re-allocated 
to higher priority users. Thus, while all units will likely incur a reduction in throughput, 
lower priority users will see a significant decrease in throughput, likely seen as less 
access to information. While this may seem like an acceptable mitigation plan, 
understand that almost all commanders, at almost every echelon of operations, have 
become accustomed to having access to a large amount of information and bandwidth. To 
think that the sudden decrease in access would occur without impact would be foolish. 

Work done by Lindquist (2004) was an example of an early attempt to capture the 
impacts of degraded communications induced by adversary use of Electronic Warfare 
(EW) assets on the combat effectiveness of the Euture Combat System (ECS), a heavily 
informationally dependent SoS architecture that was later scraped for cost overruns and 
complexity. While his work is fairly low resolution, it is one of the first attempts to try to 
quantify the operational impacts of degraded communications. In his work, Eindquist 
(2004) shows that “when communication range is degraded more than 25 percent, the 
result is nearly three times the expected number of ECS casualties” (xxi). In addition to 
this primary finding, he also makes another observation that has nearly as much 
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importance when addressing the threat from operations in a D3SOE. Lindquist observed 
that as communications are degraded, the length of battle is increased, which intuitively 
makes sense. As degradation increases, uncertainty also increases, and the natural impact 
of that uncertainty is a slower pace of battle as leaders attempt to address the uncertainty, 
and typically, there is a strong correlation between this delay and an increase in 
casualties. Thus, as Lindquist (2004) states, “even small delays (latencies greater than 
one minute) and constraints on network throughput can increase the Luture Lorce 
casualties and the duration of battle” (i). While the end state of his simulations typically 
favored the LCS force, regardless of the level of degradation, he found that “the cost of 
that victory [in lives and time] depends significantly on effective communications” 
(Lindquist 2004, v). Some of his key takeaways from this work is that it was 
advantageous for the adversary to employ its EW weapons early in the battle, the earlier 
the better, and that “enemy electronic warfare assets must not be underestimated and 
should be a focus of any pre-engagement intelligence activities” (Lindquist 2004, xxii). 
Additionally, that degradation of communications range, throughput, and latency all 
negatively impacted the combat effectiveness of friendly forces, typically observed in 
combat effectiveness as number of casualties and length of battle. Although his work 
addresses a specific problem faced by a specific system, it supports the claim that 
operations in a D3SOE will have impacts to combat effectiveness. 

b. US. ISR 

While the threats to U.S. ISR assets are different from those of U.S. SATCOM, 
the potential impacts are similar, though far more apparent for two primary reasons. Lirst, 
the U.S. has far fewer ISR assets, and thus, the loss of any capability will have a larger 
impact on the overall availability of ISR collection. Second, there are more potential 
threats to U.S. ISR assets than to U.S. SATCOM, and the risk is much greater. This is 
due to the combination of the orbital altitudes of ISR assets, which are much easier to 
attack at LEO, and the fact that adversaries continue to make advances in ASATs, 
specifically DA-ASATs and Lasers. Thus, we can expect the access to U.S. ISR 
collection assets to be put at greater risk in the future. What this means for the warfighter 

is that as combat escalates, and the adversary begins to use Lasers and DA-ASATs 
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against U.S. ISR assets, the overall availability to current high resolution imaging will be 
significantly reduced. Coupled with the dependency of ISR data to be moved via 
SATCOM links, and one can see how adversary simultaneous attacks against ISR assets 
as well as U.S. SATCOM can compound the potential threat. 

And as we saw with SATCOM, the availability of ISR collection capabilities will 
be restricted to only the highest priority users, typically at the national and strategic 
levels. This significantly reduces the availability of current ISR data to the tactical users, 
who often rely on this information to make more informed decisions as the battle 
progresses. The lack of current and timely ISR data will induce a high level of 
uncertainty into the COP, and thus, slow the speed at which the U.S. executes tactical 
operations as tactical commanders are forced to seek out other means in which to address 
informational gaps. As we saw with communications, the unrealized risk here lies with 
the fact that commanders at almost every echelon of operations have become accustomed 
to having access to a large amount collection data. Thus, as we saw before, the adversary 
“pulling the proverbial rug” out from underneath U.S. commanders will not happen 
without some operational impacts. 

Work done by Treml (2013) was an example of an early attempt to quantify the 
contributions from ISR on combat effectiveness. In his work, Treml (2013) shows that 
“the improved UAV sensor increases the situational awareness of all BLUE agents, 
which leads to a higher BLUE survivability and lethality” (74). As SA is developed, 
leaders will have a better understanding of the OE, and thus, have the information needed 
to make faster and more informed decisions. Another key finding by Treml (2013) was 
that the level of “communication and data exchange for fire coordination, targeting 
information and allocation” had direct impacts to metrics of combat effectiveness (19). 
Like Lindquist, Treml’s work links combat effectiveness to communications, information 
sharing and SA, and in his conclusion, Treml (2013) states that “the most significant 
factors for decreasing BLUE casualties are better sensors on the GCV. Better sensors 
enable the GCVs to detect, report and engage the enemy faster and to increase the 
situational awareness of all BLUE agents” (86). Again, while this only holds true to the 
scenario he developed in the context of the problem he was investigating, his findings 
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support the work of others. Some of the key takeaways from his work suggest that 
“improved sensor quality of one system combined with network capability can greatly 
enhance the performance of the whole force according to casualty expectance and battle 
duration” (Treml 2013, 88). This again highlights the importance of collection 
capabilities, as well as the necessity to disseminate that information throughout the force. 

Another key body of work was done by Soh (2013), and differed from Treml’s 
work in that it attempted to capture the impacts to combat effectiveness through the 
introduction of new capabilities, specifically the addition of ISR collection assets like 
UAVs. Her work focused on the impact that ISR systems have on SA, and how an 
increase in SA can positively impact measures of operational effectiveness like 
survivability and force protection. Her work is rooted in the SE process, and uses MBSE 
to drive the conceptual development of emerging UAV systems. In the context of my 
work, I consider this a mitigation strategy, in which we can attempt to overcome 
adversary counter-space activities through the use of additional capabilities. In her work, 
Soh (2013) shows that “outgoing communication accuracy of Infantry and Ground 
Combat Vehicles (GCV) sensor classification accuracy of Unmanned Ariel Vehicles 
(UAV), the number of UAVs, as well as UAV latency to have the most influence on 
situation awareness” (xvi). This captures two aspects of what I am attempting to address. 
Eirst, that degradation in the communication capabilities of collection assets will have a 
negative impact on combat effectiveness. This supports my earlier claims that 
degradation of collection assets, regardless if the degradation is aimed at the collection 
assets directly or at the communication paths the information takes, will have a negative 
impact on combat effectiveness. Second, increasing the number of ISR collection assets 
or their capabilities will have a positive impact on combat effectiveness. Thus, the 
introduction of additional ISR collection assets is a viable mitigation technique for 
countering adversary counter-space activities. Some of his key takeaways from this work 
is that “number of the UAVs has a significant influence over the building up of situation 
picture and its comprehensiveness” (Soh 2013, 65). Thus, the more collection assets we 
have, and the better their capabilities are regarding accuracy and speed of information 
dissemination, the faster and better our understanding of the OE will be. 
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c. 


U.S. PNT 


Because GPS jamming technology has proliferated to such an extent, GPS 
jammers have become expected in modem combat operations. All current and potential 
adversaries are expected to possess GPS jammers in some quantity at various levels of 
capability, and because of this, the U.S. military has made some progress in mitigating 
the impacts that these systems have. While there are still areas where GPS jamming 
possess a credible threat, for the most part these impacts are limited to niche areas like 
network timing disruption, deep strike fires accuracy, and weapons guidance, all of which 
are currently being resolved. Nonetheless, adversary use of GPS jammers can and will 
have an effect on operations. At the highest levels, the primary purpose of GPS jammers 
is to induce uncertainty into the COP, and thus, to slow the speed at which the U.S. 
executes tactical decision making. In an environment where GPS jammers are present, 
the warfighter will be required to put forth more effort to ensure accurate movement and 
targeting. While the length of this delay is highly dependent, there will be some amount 
of delay, and though it may seem fairly negligible, these small perturbations in decision 
making can have a much larger cumulative effect on the operation as a whole, potentially 
reduce the overall advantage that information superiority brings the U.S. military. 

Unfortunately, it is not just the PRC we should be concerned with. All state and 
non-state actors recognize the United States’ dependency on space assets as well as their 
vulnerabilities. While only a handful of potential adversaries are capable of significantly 
effecting U.S. space capabilities, all adversaries, in one way or another, are working to 
mitigate U.S. capability. 

C. THE “NEW” OPERATIONAL ENVIRONMENT 

In fact, the only specifics of the new AOC deal with its unspecific nature, no 
longer do we know who or how we will be fighting, what environment we will be 
fighting in, or with whom. In the future, the Army will “possess the ability to operate 
dispersed over wide areas because they are able to integrate intelligence and operations to 
develop situational understanding through action while possessing the mobility to 
concentrate rapidly” (U.S. Army 2014b, iii). Figure 113 shows the AOC logic chart. 
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Figure 113. Win in a Complex World Logic Chart. 

Source: U.S. Army (2014b, vi). 


As shown in Figure 113, the problem of winning in a complex world is a difficult 
one. This scenario is equivalent to studying to pass your Ph.D. oral examinations when 
the topic, the department, the location, the language, the committee members, or the time 
that the test will be given are unknown. This is a tall order and requires a solid 
understanding of not only the potential threats of the future, but also of our own 
capabilities and vulnerabilities. The problem of planning for an uncertain future is even 
further complicated by the fact that as the Army attempts to plan for an infinite set of 
potential threats and OEs, our adversaries do not. Potential adversaries have a much 
clearer and more finite understanding of their threat, the United States, and thus, do not 
suffer from the same level of uncertainty. “Countries such as China will continue seeking 
to counter U.S. strengths using anti-access and area-denial (A2/AD) approaches and by 
employing other new cyber and space control technologies” (DOD 2014b, 6). Regardless, 

371 















































the United States must face this challenge in a resource restricted environment, where the 
adversary holds many of the advantages, and prepare the force to, as the AOC states, 
“Fight and Win in a Complex World.” 

D. EMERGING MITIGATION STRATEGIES 

In the 2010 National Space Policy, the President of the United States directs the 
Secretary of Defense to “ensure cost-effective survivability of space capabilities, 
including supporting information systems and networks, commensurate with their 
planned use, the consequences of lost or degraded capability, the threat, and the 
availability of other means to perform the mission” (The White House 2010, 13). While 
this policy statement gives no specifics regarding how to accomplish the tasks, it does 
give a clear end state, which gives subordinates a clear understanding of the goal with the 
flexibility to address it as needed. Policy serves to give clear guidance to subordinates 
while providing the flexibility to investigate potential solutions without constraints and 
limitations. Take this policy as another example: “our military and intelligence 
capabilities must be prepared to ‘fight through’ a degraded environment and defeat 
attacks targeted at our space systems and supporting infrastructure. We must deny and 
defeat an adversary’s ability to achieve its objectives” (DOD 2011a, 11). The flexibility 
of the statement gives subordinates enough space to figure it out without overly 
constraining potential solutions. 

I. Doctrine 

Of all the potential areas of mitigation, emerging policy is making the most 
progress with regard to preparations for operations in a D3SOE and the establishment of 
mitigation strategies. Because policy includes top-level directives (Strategic) which flow 
down to drive TTP (Operational) development, TTPs often lag behind the establishment 
of policy. Because of this, the majority of all emerging mitigation strategies currently 
being considered are based on directives from the highest levels of National Policy, like 
the National Space Policy. These documents clearly articulate the President’s desire to 
protect our key capabilities, stating that “we will move toward less complex, more 
affordable, more resilient systems and system architectures and pursue a multi-layered 
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approach to deter attacks on space systems while retaining the capabilities to respond 
should deterrence fail” (DOD 2014b, x). Most emerging mitigation policy trends tend to 
address this statement by focusing on four broad concepts of mitigation: graceful 
degradation, resiliency, disaggregation, and cooperation. 

a. Graceful Degradation 

To win in a complex world, the overarching goal of the AOC, the Army must 
“assure uninterrupted access to critical communications and information links (satellite 
communications; position, navigation, and timing; and intelligence, surveillance, and 
reconnaissance) when operating in a contested, congested, and competitive environment” 
(U.S. Army 2014b, 32). This is concerning to me because on one hand, we are told that to 
win the United States must assure uninterrupted access to space based systems and 
capabilities in a D3SOE. On the other hand, we have also been told that adversaries “can 
and will” degrade and interrupt our space-based capabilities through the use of counter¬ 
space capabilities. Thus, we have two definitive policy statements that directly contradict 
one another. The adversary will interrupt our access to space-based capabilities, and these 
capabilities must not be interrupted for the U.S. to win. Luckily, I am not alone in 
identifying this contradiction of policy, and a relatively new way of looking at this 
problem is emerging within the space community, which is the concept of graceful 
degradation. In this concept, adversary counter-space activities will be expected to be 
somewhat effective at degrading our capabilities. Likewise, our mitigation strategies will 
also be somewhat effective at mitigating the impacts. Thus, we expect to see a gradual 
degradation of capabilities as adversaries increase the use of counter-space activities, 
somewhat mitigated by the gradual increase in United States response. The key 
difference is that by loosening the requirements away from the definitive must statements 
seen in current doctrine, we allow for enough flexibility in requirements of emerging 
doctrine to support the exploration of alternative mitigation strategies, ones that may not 
prevent an attack but are capable of reducing the impact. 
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b. Resiliency 

Because of the rapidly increasing threat and the inability of the United States to 
effectively counter it, a key emergent term that has quickly gained support from the DOD 
is the concept of resiliency. Resiliency is defined as “the ability to absorb strain and 
preserve (or improve) functioning despite the presence of adversity (both internal 
adversity ... and external adversity)” (Sutcliffe and Vogus 2003, 3). Simply put, 
resiliency is the ability of a system to maintain its operational capabilities regardless of 
friendly and enemy activities, across a wide range of possible OEs. The importance of 
resiliency in an era when the United States no longer holds a commanding technological 
advantage should be apparent. As stated in the 2014 Quadrennial Defense Review, 
investments in current and future technology acquisitions must evolve to provide more 
resilient systems and architectures for the future force to preserve the operational capacity 
of U.S. operational forces (DOD 2014b). In this strategy, policy would call for the use of 
technology, TTPs, and Army modeling to focus on improving the resilience of U.S. 
systems, and thereby reducing the impacts that potential attacks could have on U.S. 
operational effectiveness. U.S. Army TRADOC identified a set of four concepts of 
resiliency in its 2014 AOC that attempt to clarify in policy how resiliency can be 
achieved. These concepts include: the development of “resilient and hardened systems 
that degrade gracefully under attack rather than fail catastrophically”; “redundant means 
for communication and coordination”; “realistic joint training under degraded 
communications conditions”; and the pursuit of “a mix of technological and non- 
technological solutions to build sufficient redundancy and adequate reliability of 
systems.” Unfortunately, while the U.S. Army understands the value of space-based 
capabilities, as well as risk to its space-based capabilities from adversary systems, its 
implementation of the concepts of resiliency has been minimal. 

c. Disaggregation 

Another emerging means by which to mitigate the operational effects from 
operations in a D3SOE is through the concept of disaggregation by system. Simply 
speaking, disaggregation by system is the breaking down or decomposition of larger. 
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more complex space systems into smaller and simpler systems. While these smaller and 
cheaper systems are typically less capable, the loss in capability can be partially regained 
through the acquisition of an increased quantity. The advantage of disaggregation is that 
the overall risk due to the loss of any specific system is significantly reduced, and allows 
the Army to better “evade enemy attacks, deceive the enemy, and achieve surprise” (U.S. 
Army 2014b, 18). By disaggregating our space assets into numerous smaller satellites, we 
can disperse them, and increase the difficulty of our adversaries to track and target them, 
as well as reducing the operational gain achieved by doing so. To demonstrate the 
benefits of disaggregation by system, consider the loss a single WGS communication 
satellite operating over the Pacific. This loss would have a significant impact to 
operations in Theater, potentially reducing the communications capacity in theater by up 
to 33%. However, if this same capability was to be spread out among five smaller 
satellites, the loss of any single satellite would only reduce the communications capacity 
by 7%, a vast improvement. 

A variation of disaggregation by system is the concept of disaggregation by 
capabilities. Rather than building smaller versions of the same satellite, this concept 
focuses on spreading the capability out to other system alternatives. The key focus here is 
to enhance resilience through the development of “mission-effective alternatives, 
including land, sea, air, space, and cyber-based alternatives for critical capabilities 
currently delivered primarily through space-based platforms” (DOD 2011a, 11). By 
spreading the key capabilities out to other alternatives, to include satellites and other non¬ 
space systems like UAVs and near-space platforms, we can disaggregate the capability 
throughout the domain as well as into other domains, like air and near-space. 
Disaggregation by capabilities results in a decreased risk from the loss of a single system, 
as well as gives the adversary a more difficult, cross-domain targeting problem, which 
they may or may not have an available solution. 

It is for the reasons previously stated that the concepts of disaggregation have 
quickly become the buzz work within the space acquisitions communities. While the 
amount of capability lost due to employing the concepts of disaggregation remain 
unclear, especially with regard to the costs associated with the development and 
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acquisitions of these systems, it is easy to see why the concept of disaggregation has been 
receiving so much attention. Through disaggregation we can reduce risk and operational 
impacts while increasing the difficulty of the adversary to employ its space-control 
weapons...a win-win. Given the threat we currently face, and the loss of capabilities we 
have been told to expect in conflict, I believe that most commanders would be willing to 
trade off capability through disaggregation to ensure that the impact from system loss in 
conflict can be mitigated. 

d. Cooperation 

The final emergent means to mitigate the effects from operations in a D3SOE is 
through the concept of cooperation. Technically, cooperation is just another method 
disaggregation by capability, but focuses on reducing risk by spreading capability 
requirements out to other systems, specifically to commercial as well as foreign systems. 
Through cooperation the risks to military operations are reduced by spreading out space 
dependencies among commercial entities and partner nations through contracts, shared 
usage agreements, and treaties. By moving away from a single satellite provider, we 
spread that requirement out between U.S. systems, commercial systems (U.S. and 
foreign), and partner nation systems. Cooperation can multiply the effects of resiliency 
through disaggregation of capability, but with less loss in capability as seen in 
disaggregation by system because they are typically comparable to U.S. national systems. 

2. Technologies 

Counter-space systems present a significant threat to the United States’ freedom 
of maneuver, and “to prevent enemy overmatch, the Army must develop new capabilities 
while anticipating enemy efforts to emulate or disrupt those capabilities” (U.S. Army 
2014b, 11). In addressing this concern, numerous potential technological solutions have 
been considered to address the gaps identified in policy, yet the majority of these 
strategies focus on the system, and attempt to mitigate the threat primarily through 
resiliency in design, i.e., engineering and redundancy. Thus, most technological 
mitigation strategies tend to coalesce around four areas of research: the hardening of 
components from damage and component redundancy, system backups and spares, anti- 
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jam modulation technologies, and laser communications. While these technological 
mitigation strategies succeed in addressing mitigation through resiliency, they fail to fully 
account for other mitigation strategies like disaggregation. To be successful in mitigating 
the threats posed by counter-space weapons, the Army must consider all areas of 
potential mitigation, not just system level technological solutions, and “invest in and 
deliver future force capabilities to maintain a competitive advantage against increasingly 
capable and determined adversaries” (U.S. Army 2014b, 24). Fortunately, two emerging 
technological systems that seem to address both the concept of resiliency as well as the 
concept of disaggregation are SmallSats and HAAS. 

a. Small Satellites (SmallSats) 

SmallSats are emerging technologies that look to break the current trend of 
developing system level resilience through disaggregation of the system. When 
considering SmallSats, the Army should consider “deploying small affordable, 
disaggregated hosted payloads, as well as communications satellites that use more 
responsive, more affordable, space-launch services” (U.S. Army 2014a, 10). Thus, using 
SmallSats a capability like communications can be disaggregated to a constellation of 
smaller, less expensive satellites. While the overall capability of the new constellation 
may be 20-40% lower than the larger satellite, depending on the number of satellites and 
mission, the overall reduction in risk can be as high as 90-95%. Thus, for a manageable 
loss of capability, we achieve a huge reduction in operational risk. Unfortunately, the full 
implementation of SmallSats as potential mitigation strategy has been significantly 
hampered by the level of uncertainty regarding costs and capabilities. Many of these 
strategies look to be “very expensive to implement, especially considering the relative 
low costs of many ASAT systems that can destroy satellites or degrade their functionality 
in a wartime setting” (Saunders 2015, 8). Regardless, the Army, as well as the DOD has 
recognized the potential of SmallSats to address the inherent risks associated with large 
and complex satellites and is actively pursuing research in this area. By focusing more on 
technological solutions that address resilience through disaggregation of the system, the 
Army looks to better prepare itself for operations in a D3SOE. 
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b. High Altitude Atmospheric Satellites (HAAS) 

HAAS is an emergent technology that is a perfect example of disaggregation of 
capabilities, where the capabilities typically provided by a few large and expensive 
satellites, are disaggregated to other systems in other domains to reduce risk and build 
resilience. The key characteristics of HAAS that make them such an appealing mitigation 
strategy is that they operate at high altitudes, sometimes referred to as near-space, 
occupying the gray area between the Air Domain (0-18km) and the space domain (60km 
and above) which adversaries have not had to deal with before. By operating in this gray 
area between the Air and Space domains, HAAS are capable of achieving some pretty 
significant outcomes. First, they can provide nearly the same capabilities as a space-based 
platform at a significantly reduced cost. Second, they are recoverable, and payloads can 
be interchanged and tailored based on the needs of commander. Third, they are 
deployable, and thus capable of being used in support of tactical operations in 
disadvantaged areas, or as augmentation when reconstituting lost space systems. Forth, 
they are at far less risk from adversary attacks than space systems because they operate 
outside the range of most surface to air weapons, yet below the range of DA-ASATs. 
Because of these unique capabilities, HAAS have the capacity to be critical combat 
enablers of the force “when fighting though degraded space environments, while 
conducting A2/AD operations, operating in austere environments, and when surging 
forces to theater” (U.S. Army 2014a, 21). 

While I believe that HAAS represent the greatest opportunity for taking a 
proactive step to mitigate the threats faced from adversary use of counter-space weapons, 
“Army acquisition and S&T communities have been unenthusiastic in pursuing Army 
space or space-like platforms (once called ‘near-space’ platforms) such as High Altitude 
Atmospheric Satellite (HAAS) airframes, believing the responsibility of developing 
HAAS belongs to the Air Force or other joint agencies” (U.S. Army 2014a, 9). This is 
unfortunate because the Army (and Marines) is the only force concerned with the impacts 
on ground operations due to operations in a D3SOE. While the Air Force is concerned 
with operations in a D3SOE, their perspective is from the Air and Space domains, and the 
metrics they use to assess impacts do not translate to the ground force. When coupled 
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with the fact that HAAS primarily serve ground forces, fills no specific Air Force 
capability gap, operate in near-space which the Army is the proponent for, it is easy to 
see why the Air Force is unlikely to expend resources on near-space systems. 

While SmallSats and HAAS are technological solutions that can support the 
development of mitigation strategies by building system level resilience, they cannot 
ensure resilience above the system level, nor can they fully account for the concept of 
disaggregation. When considering the rate adversaries are expanding their counter-space 
capabilities, as well as the fact that “technologies change rapidly and transfer easily, the 
U.S. military will have to accelerate new technologies into the force to maintain its ability 
to overmatch enemies” (U.S. Army 2014b, 36). Thus, the United States cannot rely solely 
on mitigation strategies based on system level resilience, and as stated by TRADOC, “the 
Army’s ability to achieve significant leaps in warfighting efficiency and effectiveness 
requires an understanding of the interaction of technology with changes in doctrine, 
organizations, training, and other elements of combat effectiveness” (U.S. Army 2014b, 
36). To this end, two additional technological mitigation strategies are used in 
conjunction with the system level technological mitigation strategies to mitigate the 
impacts from operations in a D3SOE, namely mitigation through partnerships and 
mitigation through acquisitions. 

c. Mitigation through Partnerships 

In this strategy, what I like to refer to as mitigation through excess shared 
capacity, there is an abundance of space systems available at any given time that can be 
prioritized and allocated as needed. This mitigation strategy relies heavily on the 
acquisition and use of both partner nations and commercial satellite technologies to 
augment U.S. national systems. By sharing resources, the United States effectively 
expands its technological base by increasing the number of available satellites for use, 
forcing any potential adversary to re-assess the cost-benefit analysis of a counter-space 
attack. Thus, through excess shared capacity we can increase friendly capability and 
capacity while sharing the cost and risks among all participating parties. 
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d. Mitigation through Acquisitions 

In this strategy, the United States attempts to mitigate risk through the iterative 
development of current system technology to acquire newer and more capable systems in 
the future. Through continuous design improvements, the United States maintains a 
constant technological development cycle where improved and more capable systems 
across all space segments are acquired and fielded every few years. Thus, I often refer to 
it as “mitigation through acquisitions,” where we attempt to buy our way out from a 
threat. This complements mitigation through partnerships by ensuring that the U.S. 
retains some dedicated space systems that maintain the technological edge for when the 
use of partner nation or commercial satellites is not a viable solution to meeting key U.S. 
requirements. While this strategy has been successful at keeping U.S. systems at the 
forefront of space capabilities, it has done little to thwart any potential threats from 
counter-space systems. 

E. CONCLUSION 

The threat posed to U.S. Army operations from adversary space and counter¬ 
space activities is not a new vulnerability; in fact, it is well documented throughout most 
national policy and doctrine and fairly well understood. Almost all organizations 
understand that “space is vital to U.S. national security and our ability to understand 
emerging threats, project power globally, conduct operations, support diplomatic efforts, 
and enable global economic viability” (DOD 2011a, 1). However, understanding the 
threat is not the purpose of this work. The threat has already been identified, so this work 
instead focuses on understanding the impacts that this threat has on combat operations. 
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SUPPLEMENTAL 


This dissertation includes two supplemental files, which can be obtained by 
contacting the Naval Postgraduate School Dudley Knox Library. The first file is the 
Implicit Model Development Process (IMDP), which formalizes the methodology for 
expanding traditional Model Development Processes (MDPs) to address External, 
Seemingly Intangible/Non-Quantifiable (ESINQ) factors and effects when developing 
models. This contribution is codified in the form of a Microsoft Excel Spreadsheet 
consisting of 16 tabs, in addition to the nine supplemental tabs needed to address Trade 
Space Exploration (TSE). 

The second file is the Improved Relative Combat Power Assessment Tool 
(IRCPAT), which is an improved operational decision support tool developed to improve 
Army planning. The tool provides users an expanded capacity to compare the relative 
combat power of opposing forces, and to visualize and examine the impacts that external 
dependencies, ESINQ factors and effects, and external systems have on metrics of 
operational effectiveness of the ground force. Eike its predecessor, the Eorce Ratio 
Calculator (ERC), the IRCPAT is codified in the form of a Microsoft Excel Spreadsheet 
consisting of 12 primary tabs, and two example expansion tabs. 
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